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EXECUTIVE  SUMMARY 

Net-Centric  Operations  seek  to  improve  military  effectiveness  among  spatially 
distributed  and  possibly  culturally  diverse  teams.  Key  to  the  net-centric  operational 
concept  are  agile  teams  that  excel  even  in  the  most  difficult  circumstances.  Such  teams 
can  “self-synchronize,”  smoothly  coordinating  to  exploit  the  diverse  perspectives  and 
expertise  within  the  team.  The  members  of  such  teams  are  on  the  same  wavelength, 
continually  making  adjustments  to  leverage  each  others’  abilities  and  to  accommodate 
each  others’  needs. 

Recent  research  on  the  cognitive  foundations  of  collaboration  and  teamwork  has 
identified  key  cognitive  enablers  to  effective  teamwork.  These  cognitive  enablers  are  the 
knowledge  and  understandings  that  team  members  require  to  work  together  successfully. 
Accordingly,  it  is  very  desirable  that  teams  acquire  and  maintain  this  knowledge.  There 
are  three  technical  prerequisites  to  ensure  that  spatially  distributed  team  members  can  do 
this:  an  architecture  that  provides  reliable  communications  connectivity  among  team 
members,  information  that  supports  task  performance  and  team  coordination,  and  a 
means  for  team  members  to  evaluate  performance  and  correct  problems. 

This  report  describes  how  to  create  an  infrastructure  that  achieves  these  technical 
prerequisites  (Figure  1).  The  key  component  is  a  set  of  “cognitive”  node  monitors  that 
non- intrusively  help  team  members  know  how  they’re  doing.  These  monitors  help  team 
members  review  team  workload,  task  progress,  team  concepts,  and  product  quality. 
Directly  supporting  the  cognitive  node  monitor  are  two  classes  of  collaboration  tools:  1) 
those  that  help  feed  the  cognitive  node  monitors  and  2)  those  that  help  the  team  address 
any  problems  the  monitors  might  indicate.  The  architecture  provides  the  connectivity  and 
information  management  needed  to  support  net-centric  operations.  This  peer-to-peer 
agent  based  architecture  facilitates  information  flow,  helps  integrate  a  collaboration  tool 
suite  tailored  to  team  needs,  and  gathers  and  manages  the  cognitive  node  monitors. 

The  “cognitive”  node  monitors  will  enable  team  members  to  be  more  aware  of 
what  each  other  are  doing,  to  be  more  aware  of  their  task  progress,  to  understand  the 
concepts  and  issues  that  team  members  are  attending  to,  to  more  easily  identify  areas  of 
agreement  and  disagreement,  and  to  assess  the  quality  of  team  products  and  effects.  It 
will  automatically  depict  and  quantify  key  cognitive  issues  that  theory  says  are  essential 
to  effective  teamwork. 

Advanced  technologies  are  needed  to  develop  the  peer-to-peer  architecture  and  to 
create  and  update  the  cognitive  node  monitor.  The  peer-to-peer  architecture  will  build  on 
the  EBR  web-based  war  room  architecture,  developed  using  the  Cocoon  web- 
development  framework,  and  the  NPS  collaboration  architecture  employing  Groove  peer- 
to-peer  collaboration  technology  and  the  COABS  agent  management  system.  The 
generation  and  input  of  information  in  the  cognitive  monitoring  nodes  will  use  agent- 
based  analyses  of  EWALL  and  DCODE  records.  Cognitive  monitors  will  also  be 
supported  by  watching  user  databases  accesses,  by  analyzing  team  members’ 
communications  using  Latent  Semantic  Analysis  and  other  content  analysis  frameworks, 
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and  by  tapping  into  an  Earned  Value  Management  tool  used  to  integrate  performance 
management  with  schedule  and  cost  information. 


Human  Centric  Architecture: 

robust  connectivity,  tool  plug  and  play,  agent  management 


Tools  that  help 
feed  cognitive 
monitor 

•  DB  exploration 

•  Earned  Value 
Management  tool 

•  EWALL 

•  DCODE 

•  Text  and  voice 
communications 


Cognitive 

Monitor 

•  Workload  OK? 

•  Right  tasks? 

•  Needed  concepts? 

•  Good  results? 


Team 

Adjustments 

Actions  to 
improve  team 
performance 


Tools  that  help 
follow-up  monitor 
information 

•  Collab  Advizor 

•  Mind  Manager 

•  IBIS 

•  Visual  Links 

•  Text  and  voice 
communications 


Figure  1.  Infrastructure  to  Enable  Effective  Collaboration  in  Net-Centric  Operations 

The  net-centric  architecture  will  provide  several  collaboration  tools  that  help 
teams  follow-up  on  information  from  the  cognitive  monitors.  The  Collaboration  Advizor, 
developed  by  EBR  under  a  previous  SBIR,  will  help  teams  explore  and  understand  more 
fully  issues  that  the  cognitive  monitors  raise  automatically.  The  Collaboration  Advizor 
also  helps  team  members  identify  key  areas  of  agreement  and  disagreement  among  a 
broad  spectrum  of  collaboration  issues,  and  helps  them  identify  how  to  fix  problems.  The 
Mind  Manager  and  IBIS  help  teams  structure  and  discuss  problems.  Visual  Links  helps 
them  discover  and  understand  key  relationships  within  the  team  and  in  the  external 
environment. 

The  cognitive-based  architecture  and  the  novel  “cognitive”  node  monitors  will 
benefit  the  Navy,  DOD,  and  all  others  that  aspire  to  excellent  performance  by  their  teams. 
It  will  improve  team  performance,  helping  team  members  stay  on  the  same  cognitive 
wavelength  and  helping  them  adjust  and  self-synchronous  to  maintain  excellent 
performance  throughout  their  mission.  It  will  also  help  with  performance  evaluation,  not 
only  providing  data  that  teams  need  to  monitor  their  performance  and  correct 
weaknesses,  but  also  providing  critical  team  performance  data  necessary  to  evaluate  new 
collaboration  tools,  organizations,  and  processes. 
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1.  INTRODUCTION 

1. 1  Cognitive  Fragmentation 

This  report  describes  a  strategy  and  an  architecture  to  help  teams  be  more 
effective  by  avoiding  “cognitive  fragmentation.”  Cognitive  fragmentation  arises  when 
team  members  do  not  understand  how  their  work  fits  into  the  larger  team  fabric,  do  not 
know  how  perturbations  to  their  work  or  others’  tasks  will  ripple  through  the  team,  do  not 
know  how  their  understandings  and  viewpoints  align  with  others’,  and  do  not  know  how 
the  diverse  backgrounds  and  expertise  in  the  team  can  leverage  and  amplify  each  other  to 
obtain  a  superior  product  that  no  single  person  on  the  team  could  have  produced  alone. 

Cognitive  fragmentation  leads  to  many  team  problems.  Cognitively  fragmented 
teams  cannot  harmonize  their  intellects  so  that  their  many  ideas  can  be  combined  to 
create  an  outstanding  product,  do  not  know  how  to  determine  the  totality  of  viewpoints 
on  the  team,  do  not  know  where  individual  team  members  may  have  misgivings  about 
team  positions,  do  not  know  how  to  handle  unexpected  obstacles,  and  cannot  gracefully 
adjust  to  new  circumstances.  In  extreme  cases,  team  members  don’t  know  where  they  are 
going,  don’t  know  what  to  do,  don’t  talk  to  each  other,  don’t  know  how  to  get 
information,  and  don’t  know  who  they  should  listen  to. 

Cognitive  fragmentation  can  prevent  a  team  from  achieving  superior  results, 
despite  ample  resources  and  high  motivation.  Cognitive  fragmentation  can  also  cause 
serious  team  blunders.  For  example,  it  contributed  to  both  of  the  two  most  serious, 
publicized,  and  analyzed  team  blunders,  the  Bay  of  Pigs  invasion  (Janis,  1972)  and  the 
Vincennes  shoot  down  (Klein,  1998).  In  the  Bay  of  Pigs  case,  where  team  members 
approved  a  highly  flawed  plan,  team  members  did  not  share  their  individual  reservations 
because  they  did  not  wish  to  undermine  team  harmony.  Had  team  members  been  aware 
of  each  other’s  reservations,  they  might  have  discussed  them  more  fully  and  avoided  a 
calamitous  decision.  In  the  case  of  the  Vincennes  incident,  no  one  had  a  full  grasp  of  all 
of  the  team  members’  various  incomplete  and  uncertain  understandings  concerning  the 
different  clues  about  the  identity  and  behavior  of  the  approaching  airliner. 

1.2  Understanding  Cognitive  Fragmentation 

To  find  a  way  to  help  teams  avoid  cognitive  fragmentation,  we  must  first 
understand  its  cognitive  causes,  next  know  how  to  recognize  it,  and  third  know  how  to  fix 
it.  Figure  2  represents  a  teamwork  process  flow  useful  for  understanding  and  addressing 
each  of  these  issues. 

The  “teamwork  and  taskwork”  block  in  Figure  2  is  where  the  actual  collaboration 
and  work  occur.  It  has  two  inputs:  information  and  team  members’  knowledge.  The  latter 
is  labeled  “cognitive  enablers.”  These  enablers  are  the  knowledge  that  team  members 
need  to  work  together  effectively  (Noble,  2004).  Team  members  often  use  a  variety  of 
task,  collaboration,  and  communications  tools  to  help  them  carry  out  their  work. 
Observers  watching  a  team  work  can  see  the  signs  of  their  activity:  they  can  see  people 
busy  doing  work  (workload),  can  see  tasks  being  performed,  can  observe  concepts  being 
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discussed  or  addressed,  and  can  see  the  results  in  intermediate  products.  In  the  next 
block,  “evaluation,”  team  members  assess  the  adequacy  of  their  work,  plan,  and  team 
performance.  If  everything  is  OK,  then  they  continue  their  work.  If  they  recognize 
problems  and  are  a  good  team,  then  they  try  to  fix  the  problems.  Though  the  source  of  the 
problems  may  be  poor  resources  or  motivation,  this  process  flow  focuses  on  the  problems 
that  are  cognitive  because  these  are  frequently  the  major  (and  sometimes  unrecognized) 
drivers  of  team  performance.  After  the  team  fixes  their  cognitive  shortfalls,  they  continue 
their  work. 


Figure  2.  Evaluating  Team  Performance  and  Fixing  Cognitive  Collaboration 

Cognitive  enablers.  In  the  framework  of  Figure  2,  knowledge  is  central  to 
successful  teamwork  and  collaboration.  The  DTIC  report  “Understanding  and  Applying 
the  Cognitive  Foundations  of  Effective  Teamwork”  (Noble,  2004)  grouped  the 
knowledge  that  team  members  need  to  work  together  effectively  into  twelve  knowledge 
enablers.  Four  of  these  (Table  1)  are  especially  important  for  avoiding  the  problems  of 
cognitive  fragmentation. 

1.  Activity  awareness,  in  which  team  members  know  what  other  team  members  are 
working  on,  know  how  busy  others  are,  and  know  to  what  extent  what  others  are 
working  on  is  in  accordance  with  the  team’s  plans  and  team  goals. 

2.  Task  assessment,  in  which  team  members  know  what  tasks  are  being  performed, 
who  is  performing  them,  how  they  are  progressing,  what  resources  and 
information  are  needed  to  complete  them  successfully,  what  problems  are  being 
encountered,  and  what  help  should  be  offered. 
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3.  Mutual  understanding,  where  team  members  know  what  others  know  and  know 
the  extent  to  which  other  team  members  agree  or  disagree  with  each  other  and 
with  the  consensus  of  the  team. 

4.  Situation  understanding,  in  which  team  members  know  what  is  happening,  know 
the  underlying  reasons  for  why  this  is  happening,  and  can  predict  what  could 
happen  next  should  various  future  conditions  prevail.  They  know,  in  addition,  the 
evidence  supporting  their  situation  understanding,  the  signs  that  certain  events  are 
becoming  more  likely,  and  the  situation  uncertainties. 

Table  1.  Understandings  Essential  to  Team  Cognitive  Unification 


Activity 

Awareness 

Task 

Assessment 

Mutual 

Understanding 

Situation 

Understanding 

•  What  others  are 

•  Tasks  being 

•  Others’  expertise 

•  Surface 

doing 

worked  on,  and  by 
whom 

•  Others  current 

understanding 

•  How  busy  they  are 

•  Progress  towards 

understandings 

•  Situation  model 

•  Extent  to  which 
work  accomplished 

goal  achievement 

•  Extent  others  agree 
or  disagree 

•  Evidence 

adds  to  goal 
achievement 

•  Possible  problems 
and  impact 

•  Uncertainty 

Taskwork,  teamwork,  and  tools.  Taskwork  is  the  work  that  team  members  do  to 
accomplish  their  individual  tasks  assignments.  Teamwork  is  the  work  they  do  to  maintain 
their  team’s  health,  to  keep  everyone  on  the  same  wavelength,  and  to  contribute  to  team 
goals  and  objectives.  Often,  teams  use  some  task  and  collaboration  tools.  Some  of  these 
focus  on  the  specific  needs  of  a  particular  task,  such  as  a  tool  to  help  associate  track 
reports  with  tracks  or  a  tool  to  help  structure  free  text.  Others  are  more  general,  helping 
team  members  with  many  different  kinds  of  tasks.  Though  there  are  numerous 
collaboration  tools  serving  many  different  needs,  there  are  three  types  of  tools  of  special 
importance  to  addressing  cognitive  fragmentation:  1)  the  “cognitive  monitors”  that 
directly  support  several  key  cognitive  enablers,  help  monitor  critical  aspects  of  team 
performance,  and  help  teams  determine  if  the  team  is  experiencing  problems  that  need 
attention;  2)  tools  that  not  only  directly  help  with  taskwork  and  teamwork,  but  also  help 
feed  the  information  that  the  cognitive  monitors  need;  3)  tools  that  can  help  teams  fix  the 
problems  the  cognitive  monitors  detect.  Figure  1  listed  five  tools  in  the  second  and  third 
categories.  Tools  in  the  first  category  were  the  War  Room  database  exploration  tool, 
EWALL,  DCODE,  an  Earned  Value  Management  tool  for  measuring  task  progress,  and 
text  and  text  and  voice  communications.  Tools  in  the  second  category  include  the 
Collaboration  Advizor,  Mind  Manager,  IBIS,  and  Visual  Links,  and  communication 
tools. 
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Evaluation.  To  keep  the  team  on  track,  team  members  need  to  monitor  their 
performance  and  diagnose  and  fix  problems.  Skilled  teams  can  do  this  if  they  know  what 
to  look  for  to  recognize  various  team  dysfunctions.  Because  the  expertise  to  do  this  is 
often  not  available  in  a  team,  diagnosis  tools  can  help  the  team  recognize  problems.  One 
tool  that  helps  teams  do  this  is  the  Collaboration  Advizor™,  which  EBR  developed  in  a 
previous  ONR  research  program.  To  use  this  expert  system,  team  members  answer  a 
series  of  questions,  which  the  tool  then  assembles  to  create  a  “team  view”  that 
summarizes  critical  team  cognitive  issues.  This  research  will  develop  another  tool  to  help 
with  evaluation,  the  “cognitive  monitors.”  Unlike  the  Collaboration  Advizor,  which 
requires  team  members  to  answer  questions,  the  collaboration  monitors  will  run 
automatically,  tapping  into  team  information  generated  as  a  by-product  of  the  team’s 
taskwork  and  teamwork  tools. 

Fix  problems.  Once  the  team  understands  the  nature  of  their  problems,  they  can 
address  them.  While  the  collaboration  monitors  will  help  team  members  be  aware  of 
general  team  issues,  they  do  not  drill  down  to  the  precise  nature  of  the  problem  nor  do 
they  recommend  how  to  fix  these  problems.  In  contrast,  the  Collaboration  Advizor  does 
both  of  these  functions.  The  human-focused  system  includes  the  Collaboration  Advizor 
as  well  as  several  other  tools  that  help  teams  fix  problems. 

1.3  Report  Overview 

The  goal  of  the  “cognitive  architecture  for  net-centric  operations”  is  to  help  teams 
achieve  superior  results  by  1)  providing  a  set  cognitive  monitors  that  help  prevent 
cognitive  fragmentation  by  improving  team  transparency  and  helping  team  members 
detect  team  problems;  2)  providing  a  set  of  tools  that  directly  support  teamwork  and 
taskwork  and  also  enhance  the  cognitive  monitors;  and  3)  providing  an  architecture  and 
physical  infrastructure  featuring  robust  connectivity,  a  powerful  tool  set,  and  intelligent 
information  management. 

Chapters  3,  4  and  5  discuss  each  of  these  issues  in  detail. 

1.  Chapter  3  discusses  the  cognitive  monitors  that  help  team  members  be  aware  of 
the  status  of  critical  aspects  of  the  team,  and  help  alert  the  team  to  possible  team 
cognitive  deficiencies.  This  chapter  details  what  these  monitors  will  do  and 
outlines  the  technical  approach  for  non-intrusively  obtaining  the  monitoring  data. 

2.  Chapter  4  describes  a  set  of  cognitively-focused  tools  that  help  team  members  do 
their  jobs,  and  in  addition  help  the  cognitive  monitors  benefit  the  team.  This 
report  discusses  two  classes  of  such  tools.  The  first  are  tools  that  can  collect  and 
feed  information  that  the  cognitive  monitors  need  to  make  their  assessments.  The 
second  are  tools  that  can  help  teams  further  diagnose  and  then  fix  the  problems 
cued  by  the  monitors. 

3.  Chapter  5  describes  our  intelligent  agent-based  “human- focused  architecture  for 
net-centric  operations.”  This  architecture  builds  directly  on  the  existing  EBR  war 
room  and  NPS  architectures.  It  ensures  connectivity  among  a  distributed  team, 
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collects  and  manages  the  information  that  the  cognitive  monitors  need,  and 
enables  the  integration  of  tools  such  as  those  described  in  Chapter  4. 

In  addition  to  the  three  chapters  noted  above,  this  report  contains  two  additional 
sections.  Chapter  2  describes  the  cognitive  underpinnings  to  our  approach.  It  describes 
cognitive  fragmentation  more  fully,  and  explains  why  cognitive  fragmentation  is  closely 
associated  with  the  four  cognitive  enablers  listed  in  Table  1. 

Chapter  6  concludes  this  report  by  describing  how  this  cognitive-focused 
architecture  will  significantly  benefit  the  Navy  and  other  organizations  for  which 
collaboration  is  important.  It  describes  two  kinds  of  benefits:  1)  a  technical  infrastructure 
for  superior  team  performance  in  challenging  collaboration  environments;  and  2) 
instrumentation  to  support  evaluation  of  team  performance,  as  needed  to  evaluate  new 
collaboration  tools,  processes,  and  organization. 
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2.  COGNITIVE  UNDERPINNINGS 

This  chapter  reviews  the  theoretical  justification  for  our  design  of  the  human- 
centered  architecture.  It  first  describes  cognitive  fragmentation  in  more  detail,  drawing  on 
examples  used  in  previous  research  (Noble,  2004).  It  then  reviews  a  general  framework 
for  thinking  about  collaboration,  making  a  case  that  knowledge  is  central  to  effective 
collaboration.  Third,  it  suggests  the  particular  knowledge  shortfalls  responsible  for 
cognitive  fragmentation,  and  argues  that  a  system  that  can  strengthen  those  knowledge 
areas  will  enable  teams  to  avoid  this  problem. 

Most  of  the  material  in  this  chapter  was  documented  in  the  report  “Understanding 
and  Applying  the  Cognitive  Foundations  of  Effective  Teamwork”  reporting  work 
performed  under  previous  research  sponsored  by  the  ONR’s  Human  System’s 
Department.  In  addition,  it  draws  on  work  at  the  Naval  Postgraduate  School  that 
independently  documented  the  importance  of  team  awareness  to  effective  teamwork 
(Bordet sky,  2000). 

2. 1  Description  of  Cognitive  Fragmentation 

Cognitive  fragmentation  is  individual  team  understandings  that  do  not  connect. 

It’s  people  not  knowing  how  their  work  fits  with  others,  or  not  knowing  that  their 
activities  conflict.  It  is  having  inconsistent  understandings,  and  not  knowing  it. 

In  the  initial  research  described  in  the  DTIC  report,  EBR’s  subcontractor,  Klein 
Associates,  documented  20  cases  of  collaboration  shortfalls.  Unlike  the  well  studied  Bay 
of  Pigs  and  Vincennes  examples,  all  of  these  were  everyday  team  problems.  The  two 
included  in  the  DTIC  report  are  also  presented  here,  to  illustrate  cognitive  fragmentation. 

Example  1:  Performing  work  no  longer  needed  and  too  late  to  be  useful 

In  a  training  exercise,  afire  was  reported  in  a  room  that  contained  an  electrical 
box.  By  the  time  the  engineering  group  was  alerted,  the  simulated  fire  had  already  been 
extinguished  with  no  harm  to  the  electrical  system.  However,  no  one  informed  the 
engineers  of  this,  and  they  spent  the  next  hour  and  a  half  planning  no  longer  needed 
contingencies  should  the  electrical  system  be  damaged.  After  they  completed  their  plans 
and  were  about  to  present  them  to  the  entire  Technical  Support  Center,  they  discovered 
that  the  fire  was  out.  If  the  fire  had  been  real,  their  planning  would  have  come  far  too 
slowly  to  be  useful.  Yet  they  were  congratulated  on  the  quality  of  their  plans. 

This  is  cognitive  fragmentation  because  part  of  the  team  did  not  know  that  their 
work  was  no  longer  necessary,  and  those  that  did  know  did  not  realize  they  needed  to 
inform  the  others  of  this.  In  this  case,  the  Klein  Associates  reviewers  further  noted  that 
the  team  did  not  realize  that  anyone  should  have  notified  the  engineers  that  the  fire  was 
out  before  they  even  began  their  deliberations. 
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Example  2:  Taking  the  Wrong  Action 

In  a  small  company,  one  individual  had  die  job  of  keeping  die  computer  systems 
working.  One  day  a  mouse  stopped  working  that  was  attached  to  one  of  the  main 
(critical)  servers.  The  systems  operator  (sysop)  sent  a  request  to  purchase  a  replacement. 
To  make  sure  the  request  was  perfectly  clear,  he  tracked  down  when  the  original  mouse 
had  been  ordered  and  wrote  that  he  wanted  the  exact  same  mouse.  He  even  included  the 
date  of  the  earlier  purchase  order  for  reference.  He  believed  he  had  covered  all  bases. 

He  had  done  a  careful  and  thorough  job.  There  should  be  no  ambiguity. 

To  the  sysop ’s  surprise,  the  replacement  mouse  didn  ’t  work;  it  didn  7  even  fit. 
Somehow  the  front  office  has  ordered  the  wrong  one.  In  tracking  down  the  reason,  the 
sysop  found  that  the  hardware  company  no  longer  made  the  original  mouse. 

The  front  office  had  assumed  that  the  sysop  was  trying  to  indicate  the  company  he 
preferred  to  order  from.  They  contacted  that  company  and  ordered  the  mouse  closest  in 
price  to  the  original.  They  were  not  aware  that  there  was  a  compatibility  problem,  that 
not  all  mice  fit  all  machines.  To  complicate  matters  further,  the  sysop  was  traveling  when 
they  ordered  the  mouse,  so  they  could  not  ask  him.  Since  they  felt  that  he  wanted  the 
replacement  quickly,  they  did  not  wait  for  him  to  return.  They  wanted  to  show  how 
responsive  they  were. 

This  is  cognitive  fragmentation  because  the  system  operator’s  understanding  was 
inconsistent  with  the  front  office’s,  and  neither  knew  that  it  was. 

2.2  The  Importance  of  Knowledge  to  Effective  Teamwork 

The  following  four  statements  posit  the  key  premises  about  the  importance  of 
knowledge  for  effective  collaboration. 

1.  Knowledge  is  central  to  collaboration  and  teamwork.  Teams  whose  members 
know  what  they  need  to  know  can  work  together  effectively.  Those  that  do  not  are 
prone  to  various  kinds  of  predictable  errors,  with  the  type  of  error  dependent  on 
the  type  of  knowledge  deficiency.  (Liang,  1995) 

2.  Knowledge  must  be  distributed  among  members  of  a  team.  Everybody  does  not 
need  to  know  everything  for  a  team  to  be  effective.  But  every  team  member  does 
need  to  know  how  to  get  the  knowledge  he  or  she  needs.  (Wegner,  1987) 

3.  Individuals  need  to  know  about  both  “taskwork”  and  teamwork.  Teamwork 
knowledge  is  what  team  members  need  to  know  to  work  together  effectively  . 
Taskwork  knowledge  is  what  team  members  need  to  know  accomplish  their  part 
of  the  team’s  tasks.  (Canon-Bowers,  1993) 

4.  The  collaborative  dialog  helps  generate  the  needed  teamwork  and  taskwork 
knowledge.  Team  members  exchange  ideas  to  put  in  place  the  knowledge  and 
understandings  that  team  members  must  have  for  the  team  to  achieve  its  mission. 
(Argote,  2000) 
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The  first  statement,  that  team  members  cannot  work  together  effectively  if  they  do 
not  have  the  knowledge  needed  to  do  so,  is  our  basic  premise,  and  as  written  is  almost  a 
tautology.  The  second  statement,  that  team  members  do  not  personally  need  to  know  all 
critical  knowledge  but  do  need  to  know  who  to  ask  to  get  the  knowledge,  is  the  basis  of 
“transactive  memory.”  Sharing  the  responsibility  for  keeping  track  of  various  kinds  of 
information  is  one  of  the  biggest  advantages  of  teamwork.  The  third  item  emphasizes  that 
all  teams  are  really  working  on  two  basically  different  kinds  of  issues:  (1)  creating  their 
task’s  products  or  performing  task  actions,  and  (2)  maintaining  team  relationships.  It  is 
not  enough  for  every  team  member  to  be  an  expert  in  their  individual  jobs  for  the  team  to 
succeed;  team  members  also  need  to  know  how  to  work  together.  The  last  item  addresses 
how  team  knowledge  builds  on  itself.  In  teamwork,  there  is  a  kind  of  self  reinforcing 
cycle.  Knowledge  is  needed  for  teams  to  work  together  effectively,  but  teams  need  to 
work  together  in  order  to  obtain  this  knowledge. 

Figure  3  illustrates  the  relationship  between  knowledge  (the  “individual  and 
shared  understandings”)  and  some  key  team  activities:  “team  set  up  and  adjustment,” 
“group  problem  solving,”  and  “synchronize  and  act.”  This  diagram  helps  show  how 
knowledge  success  begets  success  and  small  failures  grow  into  big  ones. 


Team  Set  Up  and 
Adjustment 


Group  Problem 
Solving 


Synchronize 
and  Act 


•  Form  team 

•  Review  goals 

•  Identify  tasks 

•  Determine  roles 


•  Brainstorm 

•  Prioritize 

•  Discover  differences 

•  Negotiate 

•  Reach  consensus 


i 

Team 

Issues  to 

Discussion 

set  up 

work  on 

results 

1 

r 

Individual  and  Shared 
Understandings 


•  Mass  effects 

•  Lay  groundwork 

•  Hand  off  tasks 

•  Backup 

•  Cue  to  situation 


Performance 

feedback 


What  to 
do  next 


•  About  plan,  goals,  tasks,  and  situation 

•  About  team  members  backgrounds, 
activities,  and  status 

•  About  team  status 


Figure  3.  Relationship  between  Knowledge  and  Team  Activities 

All  teams  perform  all  three  of  these  activities,  generally  moving  from  left  to  right 
but  also  switching  back  and  forth  among  activities  according  to  their  immediate  needs.  In 
“set  up  and  adjustment”  the  team  organizes  itself,  reviewing  goals,  allocating  roles  and 
tasks,  and  defining  the  team’s  business  rules.  In  the  process  of  doing  this,  they  generate 
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and  deposit  critical  team  knowledge  about  goals,  tasks,  roles,  and  team  interaction 
methods.  Some  of  this  knowledge  may  be  written  down  in  team  documents,  but  much  of 
it  will  reside  as  tacit  knowledge  in  team  members’  minds.  Team  members  need  this 
knowledge  when  they  carry  out  their  “group  problem  solving.”  Here  they  identify  and 
critique  different  issues,  discover  differences  and  align  understandings,  negotiate,  and 
reach  consensus  about  the  nature  of  the  problem  and  what  they  should  do.  When  doing 
this,  they  draw  on  the  knowledge  acquired  while  performing  earlier  team  tasks.  As  they 
progress,  they  refine  and  augment  their  knowledge  with  the  results  of  their  work.  The 
same  sequence  also  occurs  with  “synchronize  and  act.”  Team  members  draw  on  their 
knowledge  to  coordinate  and  help  each  other.  They  deposit  knowledge  about  what  works 
well  and  how  they  should  interact  as  they  work  together. 

This  work-knowledge  relationship  can  create  highly  damaging  action-knowledge 
cycles.  A  small  amount  of  missing  knowledge  can  undermine  a  team  activity  that  creates 
information  critical  to  later  team  functions,  and  when  missing,  causes  these  later 
functions  to  fail.  Thus,  it  is  important  for  teams  to  catch  these  small  knowledge  gaps 
quickly,  before  they  grow  and  cause  significant  damage. 

2.3  The  Four  Cognitive  Enablers  Critical  for  Cognitive  Fragmentation 

Given  that  knowledge  is  critical  to  effective  teamwork,  the  next  step  is  to  specify 
what  the  knowledge  is.  This  knowledge  is  extensive,  spanning  both  teamwork  and 
taskwork,  and  incorporating  both  long  term  slowly  changing  knowledge  as  well  as 
rapidly  changing  dynamic  information.  There  are  many  different  ways  to  organize  this 
knowledge.  The  DTIC  report  divided  the  needed  knowledge  into  the  following  twelve 
“knowledge  enablers:” 

Goal  understanding.  Knowing  what  the  customer  wants,  and  knowing  the  criteria  for 
evaluating  team  products. 

Understanding  of  roles,  tasks,  and  schedule.  Knowing  who’s  supposed  to  do  what  and 
when,  and  with  what  information  and  resources. 

Understanding  of  relationships  and  dependencies.  Knowing  how  tasks  affect  each 
other  and  goals,  and  knowing  how  the  external  situation,  information,  and 
resources  impact  tasks. 

Understanding  others.  Knowing  other  team  members’  backgrounds,  capabilities, 
knowledge,  work  habits,  and  preferences  are 

Understanding  of  team  “business  rules.”  Having  and  knowing  effective  and  agreed 
upon  rules  for  team  members  to  interact  with  each  other. 

Task  skills.  Knowing  how  to  do  one’s  assigned  work,  to  include  knowing  how  use  task 
tools,  find  needed  information,  and  get  help. 

Activity  awareness.  Knowing  what  others  are  doing  now  and  the  current  need  for  doing 
it. 
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Understanding  of  the  external  situation.  Knowing  the  status  of  people  (including 

client),  things,  and  events  of  the  world  outside  of  the  team  and  projecting  future 
changes. 

Current  task  assessment.  Keeping  tasks  on  track,  knowing  how  well  own  and  other’s 
tasks  are  progressing,  and  when  to  offer  help. 

Mutual  understanding.  Knowing  what  other  team  members  understand  now  and 
knowing  if  they  agree  or  disagree. 

Plan  assessment.  Predicting  whether  the  plan  will  still  enable  the  team  to  achieve  its 
goals. 

Understanding  of  decision  drivers.  Judging  and  applying  the  criteria  for  selecting  an 
action,  and  knowing  the  decision  deadline,  how  to  manage  uncertainty,  and 
involving  the  right  people. 

Four  of  these  are  of  special  significance  to  cognitive  fragmentation.  These  are 
activity  awareness,  task  assessment,  mutual  understanding,  and  situation  understanding. 
Table  1  summarized  the  components  of  these  enablers. 

The  elements  of  activity  awareness,  task  assessment,  and  mutual  understanding 
are  straightforward  to  interpret.  Because  the  elements  of  situation  understanding  reflect  a 
cognitive  theory  of  situation  understanding,  they  warrant  further  explanation.  As  listed  in 
Table  1,  situation  understanding  has  four  components:  surface  understanding,  situation 
model,  evidence,  and  uncertainty. 

•  Surface  understanding  is  knowledge  of  those  aspects  of  the  situation  that  lie 
on  its  surface.  These  aspects  include  the  situation’s  entities,  actors,  physical 
environment,  activities,  and  events. 

•  The  situation  model  (sometimes  called  “deep  understanding”)  describes  how  a 
situation  works.  It’s  the  understandings  that  support  inferences  about 
unobserved  parts  of  the  situation  and  forecasts  about  the  future.  The  situation 
model  has  three  facets.  First,  it  specifies  the  kind  of  situation  at  a  level  of 
abstraction  suitable  for  inference  based  on  generalization;  e.  g.,  this  meeting  is 
a  kind  of  planning  meeting.  Second,  it  describes  the  typical  and  possible 
characteristics  of  that  kind  of  situation,  e.g.,  what  usually  happens  at  a 
planning  meeting.  Third,  it  describes  the  relationships  between  situation 
entities:  what  impacts  what  and  how  a  change  in  one  part  of  the  situation 
causes  change  in  another  part. 

•  The  evidence  is  the  observables  that  led  to  the  surface  understanding  and 
choice  of  situation  model. 

•  The  uncertainty  includes  uncertainties  both  about  the  surface  understanding 
and  situation  model.  There  are  three  different  kinds  of  uncertainty.  The  first  is 
imprecision,  in  which  the  values  of  the  elements  in  the  situation  are  not  known 
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exactly.  This  is  usually  caused  by  insufficient  information.  The  second  is 
equivocality,  “having  multiple  equally  plausible  interpretations  of  the  same 
information”  (Zack,  1999).  In  our  framework,  that  is  uncertainty  about  which 
situation  model  is  appropriate.  The  third  is  ambiguity,  when  it  is  “not  possible 
to  make  sense  out  situation,  regardless  of  the  available  information”  (Zack, 
1999).  This  is  an  inability  to  find  an  appropriate  situation  model,  leading  to 
uncertainty  about  all  the  situation  facets  provided  by  the  model. 

2.4  Assertions 

These  assertions  are  the  premises  for  the  design  of  the  human-focused 
architecture.  They  are: 

1.  If  a  team  has  adequate  knowledge  in  the  four  fragmentation-related 
enablers,  then  the  risk  of  cognitive  fragmentation  is  much  reduced. 

2.  If  a  team  is  having  cognitive  fragmentation,  it’s  likely  to  show  up  as  a 
reflection  of  these  cognitive  enablers 

3.  It  is  possible  to  create  a  set  of  autonomous  monitors  that  can  signal 
problems  with  these  knowledge  enablers,  and  that  cue  the  team  to  address 
underlying  problems  in  cognitive  fragmentation. 

The  following  chapter  describes  such  a  set  of  monitors.  It  describes  what  they  do, 
what  they  might  look  like,  how  they  work,  and  why  we  would  expect  them  to  address 
cognitive  fragmentation. 
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3.  COGNITIVE  NODE  MONITORS 

The  innovative  cognitive  node  monitors  are  the  centerpiece  of  our  human-centric 
architecture.  This  chapter  describes  these  cognitive  node  monitors  and  explains  the 
technology  through  which  they’ll  be  created.  It  begins  by  specifying  what  information  the 
monitors  will  provide,  and  illustrates  possible  computer  output  displays  that  depict  this 
information.  The  chapter  then  explains  how  the  monitor  information  products  support  the 
cognitive  requirements  described  in  the  previous  chapter.  The  next  section  explains  the 
technical  means  through  which  data  will  be  automatically  collected  as  by-products  of 
information  that  team  members  must  generate  to  accomplish  their  tasks  even  were  there 
no  cognitive  monitors.  This  section  also  explains  the  means  through  which  the  collected 
data  can  be  transformed  into  the  monitor  outputs.  Finally,  the  chapter  describes  a  risk 
mitigation  strategy  to  ensure  that  the  government  receives  a  high  valued  system  even 
should  some  of  the  more  ambitious  goals  not  be  fully  realized. 

Note  that  the  node  monitors  are  not  monitors  of  the  performance  of  individual 
team  members,  unless  a  team  member  is  the  only  person  at  a  node.  More  generally,  they 
indicate  the  status  of  the  component  of  a  team  at  a  particular  node. 

3. 1  Cognitive  Monitor  Output 

There  are  four  cognitive  node  monitors,  each  displaying  information  about  a 
different  aspect  of  the  work  being  performed  by  each  “node”  of  the  team.  Here  a  node  is 
one  or  more  people  working  on  the  same  task  at  the  same  place.  The  four  different  node 
monitors  address:  1)  the  extent  to  which  team  members  are  coping  with  their  work;  2) 
task  progress;  3)  concepts  and  issues  being  addressed;  and  4)  the  quality  of  intermediate 
products.  For  the  second,  third  and  fourth  items,  the  monitor  will  produce  both  basic 
status  information  (what  is  happening)  and  evaluation  information  (how  well  it’s 
proceeding  and  how  good  its  quality  is).  Table  2  summarizes  the  information  contained  in 
each  of  these  four  areas. 

Table  2.  Cognitive  Node  Monitor  Information  Products 


Monitor  Type 

Status  Information 

Evaluative  Information 

Coping 

NA 

Extent  that  team  component  is 
handling  workload  and  stress,  and 
keeping  up  with  tasks 

Task  Assessment 

Measured  progress  on  assigned 
tasks. 

Task  progress  with  respect  to 
schedule,  cost,  &  value 

Tasks  that  are  scheduled,  but  are 
not  being  addressed 

Concept  Flow 

Concepts  and  issues  being 
addressed  at  each  node 

Concepts  and  issues  being  received 
or  output  by  each  node 

Extent  that  the  team  component  is 
addressing  the  concepts  and  issues 
specified  by  the  “issues”  plan 

Intermediate 

Products 

Products  being  produced  at  each 
node 

Quality  of  these  intermediate 
products 
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Coping  monitor.  This  monitor  measures  how  busy  the  people  at  a  node  are, 
mostly  by  assessing  how  well  the  team  members  at  that  node  are  keeping  up  with  their 
tasks.  The  coping  monitor  addresses  the  first  element  of  the  “activity  awareness”  enabler, 
how  busy  team  members  are. 

Table  3  summarizes  the  three  levels  of  coping  which  the  coping  monitor  will 
infer.  It  also  summarizes  some  of  the  behaviors  associated  with  each  level  of  coping. 

Table  3.  Levels  of  Coping  Reported  by  the  Coping  Monitor. 


Level 

Description 

Associated  Behaviors 

Normal 

No  anticipated  difficulty 
achieving  goals  with  the  allocated 
resources.  Time  is  available  to 
help  others  as  anticipated  in 
baseline  plan 

Performing  some  work  not  directly  related  to 
tasks 

Meeting  measured  milestones  and  deliveries 

No  indications  of  task  or  goal  conflict. 

High 

Expect  to  achieve  all  goals  with 
allocated  resources.  However, 
there  is  little  or  no  time  to  address 
additional  tasks 

More  time  at  workstation; 

Ability  to  perform  scheduled  tasks 

Very  little,  if  any,  work  on  tasks  not  related  to 
immediate  goals 

Comments  about  “I’m  too  busy  to  do  this,”  or 
“don’t  bother  me  now” 

Overload 

Does  not  expect  to  achieve  tasked 
goals.  Additional  expertise,  time 
or  resources  will  be  required. 

More  time  at  workstation.  Working  additional 
hours. 

Not  achieving  performance  measures. 

Requests  for  additional  resources  or  additional 
expertise. 

Failure  to  perform  some  scheduled  tasks 

Task  Assessment.  The  task  assessment  monitor  shows  the  tasks  that  are  being 
worked  on,  where  the  people  at  a  node  are  in  a  task,  and  what  value  is  being  delivered  to 
the  task.  The  assessment  tool  can  provide  expected  completion  time  and  expected 
completion  cost  in  labor  compared  to  the  time  and  cost  projection  in  the  baseline.  It  also 
indicates  tasks  that  are  scheduled  but  are  not  being  worked  on. 

The  following  example  illustrates  what  the  task  monitor  monitors.  Suppose  that  a 
team  is  making  a  plan  to  help  the  victims  of  a  major  natural  disaster,  such  as  the  recent 
Tsunami.  The  team’s  task  is  to  make  a  plan.  To  create  a  plan,  the  planners  need  to 
perform  the  following  activities: 

Clarify  and  state  plan  objectives,  perhaps  by  defining  the  desired  plan  end  state 

1.  Determine  alternative  high  level  courses  of  action 

2.  Select  a  course  of  action 

3.  Determine  tasks 
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4.  Assign  resources  (personnel  and  material)  to  the  task 

5.  Create  a  task  schedule 

6.  Determine  conditions  that  may  arise  which  would  prevent  the  plan  from 
working,  and  develop  contingencies  for  these  conditions 

7.  Identify  measures  to  determine  the  value  of  the  work  performed  to  goal 
satisfaction. 

The  task  monitor  will  measure  and  report  task  progress.  It  will  also  report  the 
schedule  completion  time  and  cost  for  the  task,  if  the  planners  developed  a  detailed 
schedule  and  resource  requirements  projection. 

Concept  flow.  Concepts  refer  to  the  issues  or  factors  that  are  being  worked  on  at 
a  node.  They  are  the  factors  that  need  to  be  addressed  when  performing  a  task.  In  the  case 
of  planning  relief  after  a  natural  disaster,  the  planning  team  would  need  to  consider  the 
following  issues: 

1.  The  needs  of  the  people  at  a  particular  location.  These  are  for  food, 
shelter,  and  medicine. 

2.  The  priority  of  addressing  these  needs.  Some  are  life-threatening  and  must 
be  addressed  immediately.  There  are  others  that  delays  do  not  matter. 

3.  The  location  of  resources  for  addressing  these  needs 

4.  Means  for  transporting  resources  to  locations  where  needed 

5.  Means  to  coordinate  this  relief  effort  with  what  others  are  doing 

Note  the  distinction  between  what  the  task  assessment  monitor  measures  and  what 
the  concept  monitor  tracks.  The  former  monitors  progress  through  the  various  steps  of  a 
task.  In  contrast,  the  latter  tracks  the  concepts  or  issues  that  need  to  be  addressed  in 
carrying  out  the  task. 

The  status  information  for  the  concept  monitor  notes  what  concepts  are  being 
addressed  at  a  node.  The  evaluative  information  compares  these  concepts  with  the 
concepts  which  the  plan  specifies  need  to  be  addressed  in  completing  the  task. 

Because  the  concept  monitor  software  can  determine  the  concepts  being  imported 
and  exported  from  a  node,  it  can  create  a  map  of  the  flows  of  concepts  within  the  team. 
Thus,  the  concept  map  becomes  a  counterpart  to  the  traffic  flow  analysis  of  the  physical 
monitor.  However,  rather  than  measuring  bit  stream  volume,  the  concept  flow  monitor 
measures  the  concepts  flowing  into  and  out  of  a  node.  This  concept  map  can  then  show: 

1.  Input  to  a  node,  from  other  team  members  or  from  outside  of  the  team. 
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2.  Input  to  a  node  from  information  repositories 

3.  Output  from  node  to  information  repositories. 

4.  Output  from  node  to  other  team  members  of  to  locations  outside  of  the 
team 

In  its  evaluative  mode,  the  concept  map  can  compare  the  concepts  being  worked 
on  at  a  node  with  the  concepts  the  plan  specifies  as  important  to  work  on.  The  concept 
mode  can  also  note  the  extent  to  which  team  members  draw  on  the  expertise  within  the 
team.  If  the  team  prepares  a  “transactive  memory”  map  that  specifies  who  is  responsible 
for  various  types  of  knowledge  and  who  has  experience  and  expertise  in  various  fields, 
then  the  concept  flow  map  can  show  the  extent  to  which  team  members  draw  on  team 
expertise. 

As  will  be  discussed  in  Section  3.4,  we  anticipate  that  the  concepts  to  be 
identified  and  displayed  by  the  concept  monitor  will  be  defined  by  a  formal  ontology,  and 
recognize  using  Latent  Semantic  Analysis  and /  or  other  content  analysis  tools. 

Intermediate  products.  As  team  members  do  their  work,  they  create  products  or 
create  impacts  on  the  environment.  If  the  products  are  deposited  in  the  database,  then 
they  may  be  inspected  for  content,  and  summarized  using  the  same  techniques  as  used  for 
the  concept  node  monitor.  If  the  “products”  are  the  team’s  impact  on  the  environment, 
(e.g.,  number  of  satisfied  customers  in  a  customer  service  organization),  then  the 
“product”  must  be  extracted  from  records  about  the  state  of  the  environment. 

As  in  the  task  assessment  and  concept  flow  monitors,  the  intermediate  product 
monitors  can  have  an  evaluative  output,  provided  that  the  plan  specifies  the  criteria  for  a 
good  product.  These  criteria  can  specify  the  concepts  that  a  product  should  address.  The 
evaluative  output  can  then  note  the  number  of  these  concepts  that  are  addressed.  It  is  also 
possible  that  techniques  that  Latent  Semantic  Analysis  uses  to  evaluate  essay  question 
answers  can  help  the  intermediate  product  monitor  to  quantify  the  quality  of  the  concepts 
in  the  intermediate  products. 

3.2  Node  Monitor  Visualizations 

The  node  cognitive  monitors  summarize  team  performance  in  four  areas:  coping 
(extent  team  members  are  keeping  up  with  required  work);  task  progress,  concept  flow, 
and  quality  of  intermediate  products.  The  monitor  outputs  can  alert  the  team  to  problems 
in  any  of  these  four  areas.  In  addition,  they  can  increase  team  transparency  by  helping 
team  members  see  how  each  node  is  doing  in  each  of  these  four  areas. 

The  node  monitors  will  reside  continuously  in  a  small  window  on  the  computer 
screen,  much  as  the  physical  node  monitors  do  today  in  the  NPS  architecture  (shown  in 
Figure  17  in  Chapter  5).  Figure  4  illustrates  the  cognitive  monitor  depiction  that  may  be 
permanently  resident  in  the  upper-left  corner  of  the  screen. 
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Monitor  Alert  Summary 


Figure  4.  Cognitive  Node  Summary  in  Permanent  Display 

This  display  aggregates  performance  data  for  the  team  as  a  whole.  In  this  display, 
a  stop  light  shows  the  team  status  for  each  of  the  four  monitoring  areas.  The  stop  light  is 
green  if  all  nodes  are  performing  well  in  that  area.  It’s  yellow  if  at  least  one  node  is 
experiencing  some  difficulty.  It’s  red  if  one  or  more  nodes  are  performing  poorly. 

By  clicking  on  a  monitor  stop  light,  the  monitor  shows  the  status  of  that  monitor 
at  each  of  the  nodes,  as  shown  in  Figure  5. 


Monitor  Alert  Summary 
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This  example  is  for  a  team  planning  disaster  relief.  For  purposes  of  illustration, 
we  assume  it  has  five  nodes:  for  determining  relief  needs  in  the  southern  part  of  the 
disaster  area  and  for  the  northern  part,  for  finding  the  sources  for  needed  medical 
supplies,  for  arranging  transportation,  and  for  coordinating  with  other  relief  areas.  Here, 
node  1,  the  people  determining  the  needs  of  the  population  in  the  southern  part  of  the 
disaster  area,  is  having  significant  problems  addressing  the  concepts  that  the  plan  says  the 
people  at  the  node  should  be  addressing.  Node  4  is  having  some  difficulties.  These 
difficulties  may  be  failure  to  address  concepts,  or  failure  to  communicate  the  concept  to 
the  appropriate  people,  or  failure  to  draw  on  available  team  expertise. 

If  a  team  member  wishes  to  view  how  the  people  at  a  node  are  doing  in  each  of 
the  four  monitoring  areas,  he  may  click  on  the  vertical  bar  to  the  right  of  the  permanent 
monitor.  In  that  case,  a  display  such  as  Figure  6  would  appear.  The  node  selected  is  noted 
over  the  figure.  It’s  also  noted  in  the  vertical  box  at  the  right  of  the  figure.  Team 
members  can  directly  view  another  node  by  clicking  within  this  vertical  box. 


Node  4:  Transportation 


Figure  6.  Performance  at  a  Node  in  Each  of  the  Monitor  Areas 

If  a  team  member  wishes  to  review  the  concepts  being  addressed  at  a  node,  he  may 
click  on  the  concept  stop  light  in  Figure  6  to  bring  up  the  concept  map  shown  in  Figure  7. 
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(^)  Security  not  being  addressed 


Coordination  node  not  notified  on  transport  decisions  and  routes 


Figure  7.  Concept  Flow  at  Node  4,  Transportation  Node 

As  shown  in  this  figure,  other  nodes  have  sent  information  to  node  4  that  address 
concepts  concerning  needed  supplies  (users  would  see  specific  sources  of  information  by 
clicking  on  the  “input”  arrow),  needed  destinations  for  these  supplies,  volume  of  supplies 
needed,  and  needed  delivery  dates.  Node  4  has  extracted  from  the  database  information 
about  transport  availability  and  possible  routes,  and  has  deposited  in  the  database 
information  about  transport  and  route  decisions.  This  node  has  not  sent  this  information 
to  other  nodes. 

The  monitor  assigns  a  yellow  stop  light  to  this  node  because  these  concept  flows 
differ  from  the  plan  concept  flow  in  two  different  ways.  First,  the  node  so  far  has  not 
addressed  security  for  the  transport  of  supplies  to  the  disaster  areas.  Second,  the  node  has 
not  communicated  with  the  people  responsible  for  coordinating  with  other  relief  teams. 

3.3  Requirements  Satisfaction 

The  cognitive  monitors  help  team  members  achieve  a  common  understanding  of 
team  and  task  issues  in  two  different  ways.  First,  by  improving  team  transparency,  they 
directly  support  the  four  cognitive  enablers  (described  in  Chapter  2)  that  help  teams  avoid 
cognitive  fragmentation.  Second,  they  can  cue  team  problems  that  may  stem  from 
cognitive  fragmentation,  which  can  then  prompt  teams  to  take  corrective  actions. 

Support  to  cognitive  enablers.  Table  4  summarizes  how  each  of  the  four 
cognitive  monitors  supports  the  activity  awareness,  task  assessment,  mutual 
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understanding,  and  situation  understanding  cognitive  enablers.  This  table  distinguishes 
between  support  from  the  status  indicators  or  from  the  evaluative  indicators.  It  also  notes 
indirect  support,  where  a  team  member  can  infer  some  aspect  of  the  cognitive  enabler 
from  the  monitor  information. 

Table  4.  How  Cognitive  Monitor  Output  Support  the  Four  Fragmentation-Focused 

Enablers 


Coping  Meter 

Task 

Assessment 

Concept  Flow 
Map 

Intermediate 

Products 

Activity  Awareness 

What  others  are  doing 

Tasks 

Concepts1 

Implied  by 
products 

How  busy  they  are  -'■>  1 

Extent  in  accordance 
with  plan  and  goals 

Task  Assessment 

Tasks  being  worked 
on,  and  by  whom 

Progress 

Possible  problems  and 
impact 

Implied  by 
overly  busy 

Implied  by  poor 
progress 

Implied  by  not 
addressing 
needed  issues 

Implied  by  poor 
quality  products 

Resource  needs 

Mutual 

Understanding 

Others’  expertise 

Shown  by 
expertise  map 

Implied  by 
products 

Others  current 
understandings 

From  concepts 

Extent  others  agree  or 
disagree 

Inferable  by 

comparing 

concepts 

Situation 

Understanding 

Surface  understanding 

At  concept  level 

Inferred  from 
product  content 

Situation  model 

At  concept  level 

Evidence 

Uncertainty 

At  concept  level 

Legend: 

Support  from  Support  from 

status  indicators  evaluative  indicators 

Note  that  the  concept  monitor  can  track  only  those  concepts  or  issues  that  the 
monitors  have  been  set  up  to  recognize,  and  generally  only  at  an  abstract  level.  Thus,  if 
set  up  to  do  so,  they  can  recognize  when  team  members  are  addressing  various  aspects  of 
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a  situation.  We  do  not  anticipate  that  they  will  be  set  to  recognize  detailed  issues,  such  as 
whether  team  members  have  identified  the  precise  needs  of  people  at  a  particular  location 
in  the  disaster  relief  example. 

Cue  team  problems  that  may  stem  from  cognitive  fragmentation.  Some 
readings  of  the  cognitive  monitor  outputs,  such  as  team  work  overload,  failure  to  discuss 
needed  concepts,  and  falling  behind  in  one’s  tasks,  are  indicators  of  team  problems.  Most 
of  these  cognitive  outputs  can  have  multiple  causes,  some  of  which  are  not  related  to 
cognitive  fragmentation  and  some  of  which  are.  Table  5  shows  some  of  the  fragmentation 
behaviors  that  might  be  inferred  from  monitor  output. 

Table  5.  How  Cognitive  Monitor  Output  Cues  to  Cognitive  Fragmentation 


Monitor  Outputs  Symptomatic  of 
Fragmentation-Related  Problems 

Fragmentation-Related  Behaviors  Monitors 

Output  Might  be  Reflecting 

Coping  Meter 

Team  unable  to  keep  up  with  workload 

Not  receiving  needed  help/expertise  from  others 

Task  Assessment 

Some  scheduled  tasks  neglected 

Failure  to  work  on  tasks  others  depend  on 

Failure  of  others  to  cue  responsible  component  to 
work  on  task 

Schedule  slippage 

Node  not  receiving  needed  help/expertise  from  others 

Concept  Flow 

Called  for  concepts  and  issues 
neglected 

No  one  working  on  an  issue 

Experts  not  involved  in  working  an  issue 

Others  on  team  don’t  cue  node  to  address  concept 

Concepts  needed  by  a  node  not 
receiving  concepts  developed 
elsewhere 

No  flow  between  people  performing  tasks  and  experts 
who  can  help 

Intermediate  Products 

Some  needed  products  not  being 
produced  at  each  node 

Lack  of  needed  input  from  others  or  lack  of  awareness 
of  what  others  need 

Some  intermediate  products  are  low 
quality 

Lack  of  needed  input  from  others  or  lack  of  awareness 
of  what  others  need 

3.4  Technologies  to  Generate  Monitor  Output 

The  non- intrusive  means  to  collect  the  performance  data  that  the  monitors  need 
and  to  automatically  generate  the  monitor  output  are  important  science  and  technology 
contributions  of  this  STTR.  The  generation  of  monitor  outputs  requires  two  capabilities: 

1.  The  ability  to  collect  the  required  the  data  in  an  unobtrusive  manner  that 
does  not  increase  the  workload  of  the  team 

2.  The  ability  to  correctly  analyze  the  data  to  measure  coping  level,  task 
progress,  concepts  addressed  and  intermediate  products  at  each  team  node 

Table  6  summarizes  for  each  of  the  monitors  required  data,  sources  for  these  data, 
and  analysis  processes.  In  addition  to  the  required  data  elements  in  Table  6,  the  monitor 
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generation  processes  will  draw  on  communications  among  team  members  to  help 
estimate  team  performance  at  a  node. 

As  will  be  described  more  completely  in  Chapter  4,  four  specific  tools,  in  addition 
to  the  variety  of  communication  means,  will  contribute  data  to  the  monitor  analysis 
processes.  These  tools  are  the  war  room’s  data  base  exploration  tool,  DCODE,  EWALL, 
and  E VMS. 

Table  6.  Information  and  Processing  Requirements  for  the  Node  Monitor 


Cognitive  Monitor 
Element 

Required  data 
elements 

Where  the  data  will 
come  from 

Analysis  Processes 

Task  Assessment 

Baseline  budget  and 
schedule 

Work  Performed 

Costs  expended 
Milestones  Achieved 
Performance  Metrics 

Earned  Value 
Management  System 
(EVMS) 

Project  Management 
System 

Deliverables 

Variance,  Cost 
Performance  Index 
(CPI),  Schedule 
Performance 
Index(SPI), 

Coping  Monitor 

Scheduled  Workload 
Work  Performed 

Staff  Allocation 
Milestones  Achieved 
Performance  Metrics 

EVMS 

Project  Management 
System 

Deliverables 

SPI 

Latent  Semantic 
Analysis  (LSA) 
Information 

Extraction 

Concept  Flow 

Concepts  associated 
with  scheduled  tasks 
Concepts  contained  in 
communications  and 
work  products 

Planning  documents 
and  associated 
reference  material 
Captured  and  stored 
communications  and 
deliverables 

DCODE  and  EWALL 
Database  transactions 

LSA 

Information 

Extraction, 

Intermediate 

Products 

Intermediate 

Deliverables 

Performance  and 
quality  metrics 

Deliverables 

Planning  Documents 
Database 

LSA 

Information 

Extraction 

The  analytical  processes  are  the  means  by  which  the  collected  data  are  converted 
into  monitor  output.  There  are  two  basic  kinds  of  analysis  that  will  be  used.  The  task 
assessment  analyses  are  part  of  basic,  industry  accepted  methods  of  task  performance 
analysis  included  in  Earned  Value  Management  Systems  (EVMS).  These  include 
determination  of  actual  task  progress  as  compared  to  resource  expenditures  and  the 
variances  and  indices  that  are  essential  to  this  analysis.  More  detailed  descriptions  of  the 
ones  we  will  employ  are  included  in  the  Task  Assessment  portion  of  the  monitor. 

The  second  kinds  of  analyses  necessary  are  methods  that  automatically  analyze 
text  to  find  specific  events  or  analyze  the  information  within  the  team  communications 
and  task  products.  Two  kinds  of  analyses  are  important  for  the  node  monitor:  Latent 
Semantic  Analysis  (LSA)  and  Information  Extraction  (IE). 
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LSA  is  an  automatic  method  of  analyzing  the  semantic  information  within  the  text 
of  the  communications  and  work  products  and  comparing  that  information  to  specific 
domains  (Landauer,  Foltz  and  Lanham,  1998).  The  process  involves  “training”  the 
system  using  domain  specific  documents  and  text  and  then  correlating  documents  to  be 
analyzed  against  that  domain.  This  research  into  the  use  of  this  analytical  method  has 
shown  that  it  can  be  useful  in  determining  the  content  of  team  communications.  (Gorman, 
Foltz,  Kiekel,  Martin,  Cooke)  The  advantage  to  using  this  method  is  that  the  training  set 
and  the  input  set  do  not  have  to  use  the  same  key  words  or  phrases  to  generate  a  high 
correlation.  The  disadvantage  is  that  the  training  set  has  to  be  assembled  and  analyzed  by 
the  system  before  the  input  documents  can  be  compared  and  correlated.  The  scope  of  the 
training  set  determines  what  the  input  documents  will  be  compared  to. 

Information  Extraction  is  another  method  of  analyzing  unstructured  text  to  find 
specific  events  and  related  entities.  Rules  for  finding  the  patterns  of  entities  (people, 
places,  time,  things,  etc.)  are  developed.  These  include  grammatical,  ontological, 
orthographic,  semantic  and  specific  lists  for  finding  these  entities.  Entities  are  then 
assembled  into  patterns  to  include  specific  event  verb  groups  so  that  these  events  can  be 
detected  in  the  text.  The  advantage  to  this  method  is  that  it  requires  no  corpus  of  training 
documents  for  comparison.  The  disadvantage  is  that  the  rules  and  patterns  are  specific 
and  each  possible  variation  of  the  event  must  be  described  in  detail.  EBR  war  room 
makes  extensive  use  of  information  extraction  to  find  events  and  entities  and  we  have 
significant  experience  in  incorporating  these  systems  into  text  analysis  applications. 

Each  of  the  four  node  monitors  uses  a  combination  of  some  of  these  analytical 
methods.  The  following  describes  how  the  parts  of  the  monitor  assess  the  input  data  to 
create  the  assessment  of  team  health. 

3.4.1  Task  Assessment 

The  key  to  evaluating  task  progress  is  to  analyze  the  actual  performance  of  the 
team  in  carrying  out  the  taskwork.  The  proper  performance  and  management  of  projects 
requires  specific  reporting  that  teams  need  in  order  to  measure  and  report  on  progress. 
Both  the  task  assessment  monitor  and  coping  monitor  can  take  advantage  of  this 
reporting. 

The  specific  reporting  required  for  performance  management  includes: 

1.  Work  performed  by  task  -  how  much  time  did  team  members  spend 
working  on  a  particular  task 

2.  Work  scheduled  by  task  -  how  much  work  was  supposed  to  be  performed 
by  this  time  on  this  task  according  to  the  plan 

3.  The  cost  of  the  work  performed 

4.  The  value  of  the  work  performed  -  what  deliverables  or  milestones  were 
achieved  and  how  much  were  those  worth  toward  the  overall  completion 
of  the  task 
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These  data  elements  can  be  used  in  the  Earned  Value  Management  System 
(EVMS)  to  calculate  the  overall  performance  of  the  team  on  the  tasks  and  perform  some 
variance  analysis.  These  performance  metrics  for  the  current  work  performed  include  cost 
variance,  schedule  variance,  and  cost  performance  index: 

Cost  Variance  (CV)  =  Budgeted  Cost  of  Work  Performed  (BCWP)  -  Actual  Cost 
of  Work  Performed  (ACWP) 

Schedule  Variance  (SV)  =  Budgeted  Cost  of  Work  Performed  (BCWP)  - 
Budgeted  Cost  of  Work  Planned  to  be  performed  at  that  point  in  time 

These  variances  show  how  well  the  team  is  keeping  to  the  schedule  according  to 
the  achievement  of  performance  milestones  and  the  time  and  cost  of  achieving  these 
milestones.  Additionally,  these  same  data  elements  can  be  used  to  calculate  the  cost 
performance  index  using  the  formula: 

Cost  Performance  Index  (CPI)  =  BCWP  /  ACWP 

This  CPI  is  a  useful  measurement  to  determine  overall  performance  of  the  team. 
For  example,  a  CPI  of  0.75  indicates  that  for  every  dollar  the  team  is  spending,  they  are 
only  getting  75  cents  of  value  toward  completion  of  the  tasks. 

The  use  of  variance  and  CPI  will  use  overall  performance  on  the  tasks  as  an 
indication  of  team  performance  as  a  whole.  Generally,  teams  that  work  well  together  will 
keep  to  the  planned  budget  and  schedule.  Figure  8  below  shows  the  decision  process  that 
will  alert  the  team  to  potential  problems. 

Budgeted  Cost  and  Team  OK 


Work  Performance  Investigate 

Reasons 

Figure  8.  Task  Assessment  Process 

The  variance  analysis  is  important  to  discover  if  the  team  is  getting  too  far  ahead 
or  behind  in  any  one  task  in  the  project.  Getting  too  far  ahead  might  signal  team  burnout 
or  too  much  effort  to  one  part  of  the  project.  Getting  too  far  behind  indicates  a  host  of 
other  possible  problems  that  need  to  be  investigated  as  well. 

3.4.2  Coping  Monitor 

The  coping  monitor  is  closely  related  to  the  task  assessment  monitor  described 
above.  When  the  team  is  not  keeping  up  with  the  tasks,  one  reason  might  be  that  the 
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members  of  the  team  are  just  too  busy  to  do  all  the  work  they  need  to  do.  The  coping 
monitor  will  alert  the  team  when  there  are  indications  of  work  overload  so  that  further 
investigation  can  be  performed  to  determine  the  underlying  reason  for  the  problem. 

The  team  assessment  monitor  collected  data  from  the  project  management  and 
earned  value  management  systems  to  assess  the  task  performance.  The  coping  monitor 
can  use  the  same  data,  but  needs  to  consider  only  the  workload  aspects  of  the  team.  In 
this  way,  it  can  isolate  the  indications  of  work  overload  from  other  problems  such  as 
resource  constraints  or  external  factors. 

The  coping  monitor  will  also  benefit  by  capturing  and  analyzing  the 
communications  among  the  team  members  to  search  for  indications  of  work  overload. 
Communications  can  contribute  to  measurement  of  team  member  workload  by  capturing 
such  comments  as  “I’m  too  busy  to  talk  to  you  now.” 

Determining  the  workload  aspects  of  task  assessment  is  a  relatively  simple 
process.  Since  team  members  will  be  reporting  their  actual  work  progress,  the  coping 
monitor  can  use  the  schedule  variance  described  above  to  assess  workload  progress  on 
each  of  the  tasks.  This  variance  can  be  then  correlated  to  the  team  members  performing 
each  task  to  see  if  particular  nodes  correlate  to  schedule  problems.  If  tasks  associated 
with  a  particular  node  are  routinely  running  a  negative  schedule  variance,  this  becomes 
an  indicator  that  there  is  a  possible  overload  situation  at  that  node. 

The  other  indicator  of  overload  contained  in  the  communication  will  need  to  be 
assessed  with  automated  analysis  of  the  text.  This  will  require  several  steps  to  achieve. 

1.  The  first  step  is  to  capture  the  communication  in  structured  records.  These 
records  will  need  to  include  the  sender,  the  receiver,  the  time  and  the 
content  of  the  communication.  Capturing  text  based  communications  such 
as  e-mail  will  include  all  these  elements  and  since  e-mail  travels  through 
servers,  it  is  possible  to  perform  analysis  on  the  entire  corpus  of  e-mail 
communications.  Voice  communications  can  be  captured  using 
transcription  technologies  to  convert  the  voice  to  text. 

2.  The  second  step  involves  setting  up  the  system  to  recognize  the 
indications  of  work  overload  and  set  the  thresholds  for  when  this  indicates 
a  problem  that  needs  to  be  addressed.  The  thresholds  might  include  the 
overall  density  of  overload  statements,  the  concentration  of  these 
statements  to  particular  nodes  or  the  degree  of  severity  of  the  statements. 
LSA,  keyword  and  concept  searching  and  rule  based  event  extraction  are 
all  viable  methods  of  recognizing  the  indications  of  work  overload. 
Research  performed  by  New  Mexico  State  University  in  the  use  of  LSA  in 
measuring  the  content  of  team  communications  supports  the  efforts  in  this 
part  of  the  cognitive  node  monitor.  (Nancy  Cooke  and  NMSU).  The  key  to 
success  will  be  ensuring  that  a  sufficiently  large  corpus  of  documents  can 
be  assembled  to  properly  “train”  the  LSA  system,  or  that  concepts  and 
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events  related  to  work  overload  problems  can  be  identified  so  that  other 
analysis  tools  can  be  used. 

3.  Once  the  data  is  captured  and  the  analysis  system  is  setup,  the  next  step 
will  be  to  perform  the  comparison  of  the  captured  communications  with 
the  indications  of  work  overload  that  have  been  set  up  in  the  system. 

These  indications  will  generate  an  alert  that  can  be  compared  to  a  set  of 
metrics  to  drive  the  coping  monitor  and  alert  the  team  to  a  potential 
problem. 

The  output  alerts  from  the  communications  content  analysis,  when  combined  with 
the  workload  and  schedule  variance  data  from  the  task  assessment,  provides  the  complete 
data  requirements  for  the  coping  monitor.  Figure  9  shows  the  flow  chart  of  how  these 
work  together  to  create  this  monitor. 

The  analyses  of  how  well  the  individual  node  is  performing  on  its  tasks  and  how 
they  are  keeping  up  will  alert  the  team  to  potential  problems.  It  will  usually  be  necessary 
to  follow-up  investigation  to  determine  the  cause.  Chapter  4  describes  tools  that  can  help 
the  team  diagnose  and  fix  the  problem. 


E-mail  LSA  Thresholds 


Thresholds 


Figure  9.  Coping  Monitor  Process 


3.4.3  Concept  Flow 

The  concept  flow  monitor  identifies  the  concepts  that  are  being  addressed  at  a 
node  or  communicated  among  the  nodes,  and  compares  these  concepts  to  those  that 
should  be  discussed  based  on  the  project  plan  and  distribution  of  expertise  in  the  team. 

The  three  major  aspects  of  the  concept  flow  monitor  are  1)  the  determination  of 
the  concepts  that  should  be  discussed  as  specified  by  the  plan,  2)  the  assessment  of  the 
information  being  analyzed  by  the  various  experts  on  the  team,  and  3)  the  assessment  of 
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the  content  of  the  communications  among  the  team  and  the  experts  as  compared  to  the 
plan.  In  contrast  to  the  physical  node  monitor  that  analyzes  the  amount  of  traffic  between 
nodes,  the  concept  flow  monitor  will  analyze  the  content  of  that  communication  to 
determine  the  flow  of  concepts  among  nodes  and  the  extent  to  which  this  flow  is  in 
accordance  with  what  the  plan  expects. 

The  first  step  to  generating  the  concept  flow  monitor’s  output  is  to  determine  the 
concepts  that  the  plan  expects  should  be  discussed  at  various  times.  When  a  team  plans 
properly  for  a  task,  it  develops  various  planning  documents  that  define  the  scope  of  work, 
the  tasking  for  each  member  of  the  team,  the  schedule  for  that  tasking,  the  goals  and 
objectives  and  many  other  things  that  determine  what  should  be  talked  about  and  when  it 
should  be  talked  about.  It  is  these  documents,  as  well  as  the  reference  materials  that  have 
been  used  to  develop  them,  that  will  serve  as  the  baseline  of  the  concept  flow  plan. 

The  LSA  process  will  then  take  these  documents  and  create  a  domain  specific 
semantic  map  that  changes  over  time.  This  map  determines  what  concepts  should  be 
discussed,  by  whom,  and  when.  It  will  also  serve  as  the  baseline  for  determining  if  the 
team  is  receiving  the  correct  external  data  at  the  right  time. 

Once  this  domain  specific  concept  map  is  completed,  it  can  be  used  as  the 
baseline  to  compare  the  input  documents  and  the  content  of  communications.  The 
concept  map  can  be  used  to  create  a  database  of  specific  tags  to  be  compared  with  the 
input  data  so  that  the  input  data  can  be  categorized  and  evaluated.  The  concept  flow 
monitor  will  make  two  kinds  of  comparisons.  It  will  compare  the  concepts  in  the 
communications  in  and  out  of  the  node  with  the  concepts  specified  by  the  plan.  It  will 
also  assess  whether  the  node  is  receiving  the  correct  information  from  the  various 
information  repositories. 

The  physical  architecture  will  capture  the  communications  in  and  out  of  any  node 
and  from  the  information  repositories.  This  will  create  another  repository  of  information 
that  will  be  used  in  the  LSA  process  to  determine  whether  the  content  is  in  accordance 
with  the  plan.  Tools  such  as  DCODE  and  EWALL  will  also  serve  to  collect  and  store  the 
information.  DCODE  will  be  used  by  experts  to  collect  and  categorize  the  information 
received  from  outside  the  team  so  that  the  correct  part  of  the  team  can  use  it.  EWALL  can 
be  used  to  collect  and  store  the  current  work  products  and  ideas  that  are  being  discussed 
by  each  member  of  the  team.  A  map  will  then  be  created  and  the  comparison  to  the  plan 
map  can  be  made.  If  the  comparison  does  not  correlate  within  the  determined  thresholds, 
then  the  team  can  be  alerted.  Figure  10  shows  the  flow  chart  of  how  these  will  work 
together  to  create  this  part  of  the  node  monitor. 
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Figure  10.  Concept  Flow  Monitor  Process 

This  concept  flow  monitor  is  a  critical  piece  in  the  cognitive  assessment  of  the 
team’s  health.  It  will  ensure  that  the  team  is  both  communicating  and  discussing  the  right 
things  at  the  right  time.  It  will  also  indicate  the  level  to  which  the  team  is  taking 
advantage  of  the  experts  on  the  team.  Since  the  experts  will  be  the  ones  receiving  the  data 
and  categorizing  it  with  DCODE  for  input  to  the  LSA  process,  the  concept  flow  monitor 
will  then  indicate  whether  or  not  they  are  participating  in  any  further  discussions  about 
those  concepts. 

3.4.4  Interim  Products 

This  part  of  the  monitor  leverages  the  concept  flow  monitor  and  adds  another 
assessment,  the  determination  of  the  overall  quality  of  the  interim  work  products.  There 
are  two  basic  steps  to  making  this  part  of  the  monitor  work.  Metrics  of  product  quality 
have  to  be  determined  and  comparisons  of  the  work  products  against  those  metrics  have 
to  be  made. 

Determining  product  quality  depends  on  the  nature  of  the  interim  products.  They 
might  include  measures  such  as  the  extent  to  all  the  correct  concepts  are  in  the  document, 
the  level  of  completion  to  which  they’re  discussed,  and  are  the  extent  to  which  the  correct 
members  of  the  team  participating  in  the  creation  of  that  product. 

The  LSA  process  is  capable  of  determining  the  quality  of  work  products.  Tools 
that  employ  LSA  are  already  being  used  to  do  automatic  grading  of  essay  questions  in 
standardized  tests.  That  capability  can  be  leveraged  to  determine  the  overall  quality  of 
interim  work  products. 

There  are  three  main  steps  in  determining  the  overall  quality  of  the  interim  work 
products.  The  first  will  be  to  collect  or  create  the  necessary  reference  documents  to  train 
the  LSA  system  for  an  assessment  of  quality  and  completeness.  These  documents  will  be 
drawn  from  planning  documents  that  describe  the  tasks  and  their  overall  goals  and 
objectives.  The  training  of  the  LSA  system  can  include  both  analysis  and  determination 
of  the  overall  domain  for  a  complete  document,  and  also  a  tagging  process  that  will 
identify  the  main  concepts  and  compare  the  work  products  to  the  set  of  tagged  documents 
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to  ensure  that  all  the  various  concepts  are  included  (Gorman,  Foltz,  Kiekel,  Martin, 
Cooke). 

Once  the  training  of  the  system  is  complete,  the  next  step  is  to  capture  the  interim 
products  for  analysis.  The  interim  products  that  will  be  used  for  this  analysis  can  be 
stored  in  EWALL  as  well  as  the  product  database. 

Finally,  an  analysis  of  the  work  products  comparing  them  to  the  reference 
documents  using  the  LSA  system  will  show  the  correlation  to  the  semantic  space  defined 
in  the  reference  documents,  as  well  as  a  mapping  to  the  tag  database  for  key  concepts. 
Figure  11  is  a  flow  chart  showing  this  process. 
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Figure  11.  Interim  Product  Monitor  Process 

The  output  of  this  monitor  will  enable  to  the  team  to  track  the  overall 
completeness  and  quality  of  the  output  of  the  team.  This  is  of  course  one  of  the  key 
metrics  to  team  health.  If  the  team  is  creating  high  quality  products,  then  that  is  an 
indication  that  they  are  performing  well. 

3.5  Risk  Assessment  and  Management 

The  development  of  the  node  monitors  is  the  key  innovative  development  in  this 
research  program.  As  such  it  has  development  risks  and  potential  payoffs.  Our 
development  process  will  include  fallback  positions  to  ensure  that  the  monitors  produce 
information  that  will  be  valuable  to  detecting  and  understanding  team  health  issues  even 
should  some  of  the  more  ambitious  technology  developments  prove  infeasible. 

The  parts  of  the  monitor  with  the  highest  development  risks  include  the  automated 
assessments  of  the  communications,  work  products  and  deliverables.  We  are  mitigating 
this  risk  by  taking  the  following  measures: 
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•  Using  text  analysis  tools  and  technologies  that  have  reached  a  high  degree 
of  development  maturity  such  as  LSA  and  Information  Extraction  tools 

•  Using  industry  standard  software  integration  and  database  integration 
methods  to  reduce  integration  risk 

•  Using  other  tools  that  are  currently  being  used  in  other  programs  and 
projects  such  as  EWALL  and  DCODE 

•  Where  automated  collection  or  analysis  is  planned,  developing  fallback 
positions  that  will  minimize,  but  not  necessarily  eliminate  team  interaction 
with  the  monitor 

As  a  result  of  this  risk  management  strategy,  EBR  and  NPS  are  confident  that  this 
effort  will  produce  a  workable  set  of  cognitive  monitors  that  can  improve  team 
transparency,  reduce  cognitive  fragmentation,  and  alert  to  cognitive  problems. 

In  addition,  because  we  are  basing  our  architecture  on  existing  capabilities  at  EBR 
and  NPS,  there  is  no  risk  that  we  will  fail  to  create  a  highly  flexible  and  robust 
architecture  able  to  support  net-centric  operations.  Finally,  because  we  are  integrating 
available  GOTS  and  COTS  tools,  there  is  no  risk  that  the  tools  will  not  provide  their 
intended  support. 
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4.  SUPPORTING  TOOLS  AND  DOCUMENTS 

4. 1  Tool  and  Reference  Documents 

This  chapter  examines  the  tools  and  reference  documentation  that  support  the 
special  functionality  the  human-focused  architecture  (Figure  12).  The  tools  focus  on  three 
functions:  1)  preventing  cognitive  fragmentation  by  supporting  the  activity  awareness, 
task  assessment,  mutual  understanding,  and  situation  understanding  cognitive  enablers;  2) 
providing  data  to  the  cognitive  monitors;  and  3)  helping  follow-up  on  the  problems  that 
the  monitors  cue.  The  reference  documents  to  be  discussed  here  support  the  first  two  of 
these  functions:  countering  cognitive  fragmentation  and  providing  data  that  the  cognitive 
monitors  need. 


Feedback 


Figure  12.  Roles  of  Tools  and  Plans  in  Human-Focused  Architecture 

This  chapter  has  two  main  parts:  the  first  discussing  tools  and  the  second 
discussing  the  reference  documents.  The  tool  discussion  details  the  properties  that  tools 
should  have  to  support  the  three  functions  stated  above.  It  then  describes  a  few  of  the 
specific  tools  selected  for  integration  into  the  human-focused  architecture.  The  reference 
document  discussion  reviews  how  these  documents  support  the  cognitive  monitors. 

4.2  Tools  for  the  Human-Focused  Architecture. 

This  section  describes  the  capabilities  that  tools  need  in  order  to  counter  cognitive 
fragmentation,  provide  data  to  the  cognitive  monitors,  or  help  team  members  solve  the 
team  problems  cued  by  the  monitors.  There  are  numerous  specific  tools  that  can  meet 
these  requirements.  Section  4.2.2  describes  eight  that  EBR  and  NPS  plan  to  integrate  into 
the  human-focused  architecture. 
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4.2.1  Desired  Tool  Capabilities 

Tool  Capabilities  that  Counter  Cognitive  Fragmentation 

Cognitive-fragmentation  results  from  deficiencies  in  activity  awareness,  task 
assessment,  mutual  understanding,  or  situation  understanding.  Each  of  these  knowledge 
enablers  has  several  dimensions,  each  of  which  can  be  addressed  by  several  different 
types  of  tools.  Tables  7-10  describe  some  of  the  many  tool  capabilities  which  can  reduce 
the  risk  of  cognitive  fragmentation. 

Table  7.  Tool  Capabilities  that  Support  Activity  Awareness 


Cognitive  Category 

Desired  Tool  Capabilities 

1 .  Knowing  what  others  are 
doing 

•  Provide  team  transparency,  so  that  remotely  located  team 
members  may  “look  over  the  shoulder”  of  people  as  they  do 
their  jobs. 

•  Enable  geographically  dispersed  people  to  review  team 
member  products  as  soon  as  they  are  produced,  e.g., 
common  data  repositories 

•  Post  what  team  members  are  working  on;  e.g.,  task  status 
boards. 

2.  Knowing  how  busy  they 
are 

•  Show  task  status,  so  that  people  see  whether  others  are 
getting  behind  in  performing  their  tasks 

•  Provide  team  transparency,  to  help  people  see  what  others 
are  doing,  and  so  see  how  busy  they  are 

3.  Knowing  extent  in 

accordance  with  plans  and 
goals 

•  Depict  the  plan,  showing  the  schedule  for  tasks  and  the 
people  assigned  to  these  tasks.  Should  include  planned 
contingencies,  and  the  circumstances  under  which  these 
contingencies  should  be  implemented 

•  Describe  task  goals,  and  describe  characteristic  of  products 
that  support  these  goals. 
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Table  8.  Tool  Capabilities  that  Support  Task  Assessment 


Cognitive  Category 

Desired  Tool  Capabilities 

1.  Tasks  being  worked  on 

•  Post  tasks  team  members  are  working  on;  e.g.,  task  status 
boards  (can  show  people  working  on  no  task) 

•  Post  tasks  that  need  to  be  done  with  people  who  are  working 
on  (shows  neglected  tasks) 

•  Couple  transparency  tools  that  help  others  know  what  people 
are  doing  with  task  descriptions  that  specify  the  work 
required  to  perform  the  task. 

•  Couple  product  repositories  that  show  what  people  are 
producing  with  descriptions  of  the  products  to  be  produced 
by  each  task 

2.  Progress 

•  Post  work  completed. 

•  Provide  access  to  repository  of  intermediate  products,  thus 
enabling  review  of  products  as  parts  are  completed. 

•  Enable  communications  announcing  completion  of  task 
elements 

•  Describe  total  work  to  be  performed 

•  Compare  estimated  time  to  completion  with  scheduled  time 

•  Couple  estimates  of  task  status  and  work  accomplished  with 
specifications  of  criteria  for  high  quality  product 

3.  Possible  problems  and 
impacts 

•  Couple  written  plan  describing  resources  and  information 
required  to  perform  task  with  postings  showing  available 
resources  and  information 

•  Compare  estimated  time  to  completion  with  scheduled  time 

•  Describe  criteria  for  a  good  quality  product,  and  specify 
information  and  resources  required  for  the  product 

•  Couple  above  with  postings  showing  available  information 
and  resources 

•  Show  dependency  among  tasks  and  goals 

2.  Resource  needs 

•  Show  plan  describing  information,  equipment,  tools,  and 
people  required  to  perform  task 

•  Couple  plan  depiction  with  listings  of  available  and  assigned 
information,  equipment,  tools,  and  people 
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Table  9.  Tool  Capabilities  that  Support  Mutual  Understanding 


Cognitive  Category 

Desired  Tool  Capabilities 

1.  Others’  expertise 

•  Summarize  backgrounds  and  areas  of  expertise 

•  Promote  transparency  so  that  all  team  members  can  observe 
capabilities  as  people  work 

2.  Others’  current 
understandings 

•  Allow  or  help  people  to  explain  and  illustrate  their  views 
about  the  spatial,  temporal,  and  organizational  aspects  of  a 
situation. 

•  Help  people  explain  their  positions:  e.g.,  the  views  they 
advocate  and  the  rationale  for  these  views. 

•  Help  structure  the  dialog  and  environment  for  obtaining 
ideas 

•  Help  ensure  that  all  voices  are  heard 

•  Reduce  possible  perceived  risks  that  people  may  have  about 
sharing  their  views 

•  Make  available  a  means  to  review  information  available  to 
others,  to  include  access  to  the  information  feeds  and  data 
sources  that  others  use 

3.  Extent  others  agree  or 
disagree 

•  Couple  means  to  know  others  backgrounds  and  expertise 
with  means  to  perceive  or  infer  their  current  understandings 
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Table  10.  Tool  Capabilities  that  Support  Situation  Understanding 


Cognitive  Category 

Desired  Tool  Capabilities 

1.  Surface  Understanding 

•  Show  location  and  identity  of  objects  along  with  contextual 
factors  relevant  to  situation-based  decisions 

•  Show  events  on  timeline 

•  Show  relationships  among  situation  entities,  to  include  what 
elements  belong  to  the  same  organization  and  the  nature  of 
the  relationships  among  these  elements 

•  Summarize  changes  to  above 

•  Show  changes  to  above  over  time 

2.  Situation  Model 

•  Show  relationships  among  situation  entities,  to  enable 
viewers  to  know  how  various  entities  impact  one  another 

•  Show  default  values  for  a  situation  with  uncertainties,  when 
observed  values  not  available 

•  Project  to  future  possible  states  that  could  evolve  from  a 
current  situation,  and  state  conditions  under  which  these 
states  would  be  most  likely  to  arise 

•  List  the  observables  that  indicate  whether  a  particular 
situation  interpretation  (selection  of  situation  model)  is 
correct  or  not  and  can  indicate  which  possible  future  states 
are  coming  to  pass 

3.  Evidence  Understanding 

Evidence  audit  trail 

•  Show  the  evidence  used  to  generate  the  values  of  entities  and 
events  shown  on  the  situation  pictures 

•  Highlight  or  list  the  factors  responsible  for  the  choice  of 
situation  model 

•  Show  estimated  reliability  for  the  evidence,  and  reasons  for 
this  reliability  estimate 

4.  Uncertainty 

Understanding 

•  Convey  uncertainty  about  the  elements  in  the  spatial, 
temporal,  and  organizational  views 

•  Show  uncertainty  about  the  situation  relationships  and 
default  values  for  situation  elements 

•  Show  uncertainty  about  the  evidence  supporting  situation 
inferences 

•  Show  uncertainty  about  the  “type”  of  situation  being 
encountered 

Tool  capabilities  that  support  monitor  data  input  needs 

The  cognitive  monitors  indicate  levels  of  workload,  task  status  and  progress, 
cognitive  flow,  and  product  quality.  These  monitors  are  designed  to  generate  their 
estimates  autonomously,  using  data  that  already  resides  within  the  team  that  they  can  tap. 
Table  11  shows  some  characteristics  of  the  data  that  each  of  the  monitors  need. 
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Table  11.  Monitor  Data  Requirements 


Monitor 

Desired  Data  to  be  Provided 

Coping 

•  Shows  extent  keeping  up  with  schedule 

•  Shows  performance  of  non-essential  tasks 

•  Shows  failure  to  perform  some  scheduled  tasks 

•  Comments:  “I'm  too  busy  to  talk  to  you  now” 

Task  Assessment 

•  Shows  tasks  complete,  being  worked  on,  progress,  comparison 
with  plan 

•  Comments  such  as  “I’ll  never  finish  this  work  on  time” 

Concept  Flow 

•  Implies  kinds  of  issues  being  addressed 

•  Implies  inflowing  and  outgoing,  and  to  whom 

•  Shows  issues  communications  address 

•  Shows  issues  being  deposited  into  databases 

•  Shows  match  between  people  who  contribute  to  concepts  and 
expertise  on  the  team 

•  Shows  match  between  issues  being  addressed  and  what  should 
be  addressed 

Intermediate  Products 

•  Shows  types  of  products  being  deposited  into  databases  or 
transmitted  to  others 

•  Shows  issues  that  have  been  addressed  in  these  products 

•  Enables  comparison  of  issues  addressed  with  specification  of 
issues  that  should  be  addressed 

In  order  to  support  the  data  needs  of  the  monitors,  tools  need  to  generate  these 
data.  There  are  four  types  of  tools  that  are  able  to  do  this:  task  aids,  program  management 
tools,  databases,  and  communications. 

Task  aids  are  tools  that  help  team  members  collect,  organize,  and  manage 
information.  The  task  aids  that  best  support  the  monitors  are  those  that  can  provide  data 
about  the  kind  of  information  a  person  is  working  on  and  how  that  person  is  evaluating  or 
using  the  information. 

Program  management  tools  can  track  tasks  and  estimate  variance  between 
actual  and  planned  progress.  The  ones  needed  here  can  provide  data  for  tracking  and 
evaluating  progress. 

Structured  databases  (including  structured  web  sites)  label  the  data  they  contain. 
For  example,  a  database  on  company  employees  might  specify  an  employee’s  name, 
position,  salary,  and  supervisor.  In  a  database,  each  of  these  items  is  paired  with  a  role. 
Thus  a  person’s  salary,  the  number  $42,000  would  be  paired  with  the  label  “salary.” 
Because  of  this  data  labeling,  it  is  possible  to  determine  what  kinds  of  data  a  team 
member  is  working  with  when  he/she  accesses  the  database.  Databases  that  allow 
monitoring  of  the  data  fields  being  accessed  are  useful  both  for  inferring  concepts  being 
addressed  and  identifying  intermediate  products  being  worked  on  and  deposited. 
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Electronic  Communications  such  as  e-mail,  and  chat  can  potentially  be  captured 
as  free  text  flows.  Voice  communications  can  also  be  captured  and  converted  into  free 
text.  The  content  of  unstructured  databases  are  unlabeled  text.  When  analyzed  using 
content  analysis  tools  such  as  latent  semantic  analysis  or  information  extraction,  these 
texts  can  convey  the  concepts  team  members  are  addressing  or  talking  about.  These 
analyzed  texts  can  potentially  support  any  of  the  monitors.  For  example,  an  e-mail  stating 
that  “I’m  too  busy  to  meet  with  you  today”  when  analyzed  is  useful  for  inferring  a  team 
member’s  level  of  busyness.  The  information  accessed  from  unstructured  databases  can 
also  provide  this  information. 

Properties  of  tools  that  help  teams  follow  up  on  monitor-cued  problems 

The  cognitive  monitors  cue  general  problems:  difficulty  accomplishing  all  the 
work  that  needs  to  be  done,  not  keeping  to  the  task  schedule,  failing  to  address  key  issues 
or  not  drawing  on  the  team’s  expertise,  and  not  creating  some  needed  intermediate 
products.  When  the  monitor  cues  any  of  these  general  problems,  the  team  may  wish  to 
follow  up  in  order  to  determine  more  precisely  the  nature  of  the  problem  and  to  identify 
possible  remedies.  The  monitor  follow-up  tools  help  them  do  this. 

Tools  useful  for  monitor  follow-up  can  help  the  team  diagnose  the  issues  and 
identify  solutions  in  one  of  four  different  ways.  They  can  provide  diagnostic  tests  to 
pinpoint  the  problem,  can  suggest  remedies,  can  help  structure  discussions,  and  can  help 
team  members  examine  and  critique  their  understanding  of  the  situation. 

4.2.2  Overview  of  Selected  Tools’  Contributions 

There  are  numerous  tools  available  that  help  counter  cognitive  fragmentation, 
could  provide  data  to  the  cognitive  monitors,  and  can  help  teams  follow  up  monitor  cues 
of  team  problems.  This  section  describes  eight  specific  tools  that  we  intend  to  integrate 
into  our  human-focused  system.  For  completeness  we  also  discuss  communication  tools 
as  a  generic  class,  which  we  of  course  also  expect  to  include  in  the  system.  All  of  these 
tools  can  help  teams  avoid  cognitive  fragmentation.  Four  of  the  specific  tools  seem 
especially  well  suited  for  providing  data  to  the  cognitive  monitors,  and  four  can  excel  at 
helping  teams  follow-up  on  monitor  cues  to  problems.  Communication  tools  both  help 
feed  the  monitors  and  enable  team  embers  to  discuss  problems. 

The  eight  specific  tools  are  available  commercially  at  COTS  or  from  the 
government  as  GOTS.  The  first  four  are  the  war  room’s  database  exploration  tools, 
DCODE  for  organizing  summaries  of  documents,  EWALL  for  organizing,  managing  and 
sharing  information,  and  the  COBRA  earned  value  management  system.  The  second  four 
are  the  tools  selected  especially  to  support  follow-on  discussions  that  address  potential 
problems  cued  by  the  monitors. 

Table  12  summarizes  how  each  of  the  selected  tools  can  help  reduce  cognitive 
fragmentation.  Tables  13  and  14  then  summarize  how  the  tools  either  input  data  that  the 
monitors  need  or  help  teams  follow-up  on  the  monitor  output. 
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Table  12.  How  Selected  Support  Cognitive  Enablers  that  Reduce  Fragmentation 


Tools  that  Provide  Data  To 
Monitors 

Tools  that  Help  Follow-Up 
Monitors 

Both 

DB 

Explr 

DCO 

DE 

EWA 

LL 

COBR 

A- 

EVM 

Callab 

Advz 

Mind 

Mgr 

IBIS 

VL 

Com 

ms 

Activity  Awareness 

What  others  are 
doing 

How  busy  they  are 

Extent  in  accordance 
with  plan  and  goals 

Task  Assessment 

Tasks  being  worked 
on,  and  by  whom 

Progress 

Possible  problems 
and  impact 

Resource  needs 

Mutual 

Understanding 

Others’  expertise 

Others’  current 
understandings 

Extent  others  agree 
or  disagree 

Situation 

Understanding 

Surface 

understanding 

Situation  model 

Evidence 

Uncertainty 
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Table  13.  Summary  of  How  Selected  Tools  Help  Provide  Data  to  the  Monitors 


DB 

Explr 

DCODE 

EWALL 

COBRA- 

EVM 

Comms 

Coping 

Extent  keeping  up  with  schedule 

Failure  to  perform  some  scheduled  tasks 

Comments:  “I’m  too  busy  to  talk  to  you 
now” 

Task  Assessment 

Tasks  complete,  being  worked  on  and  by 
whom,  and  comparison  with  plan 

Comments  such  as  “I’ll  never  finish  this 
work  on  time” 

Concept  Flow 

Kinds  of  issues  being  addressed 

Issues  inflowing  and  outgoing,  and  to  whom 

Issues  communications  address 

Issues  being  deposited  into  the  database 

Intermediate  Products 

Types  of  products  being  deposited  into 
database  or  transmitted  to  others 

Issues  that  have  been  addressed  in  these 
products 

Table  14.  Summary  of  How  Selected  Tools  Help  Provide  Data  to  the  Monitors 


Collab 

Advzr 

Mind 

Mgr 

IBIS 

VL 

Comms 

Diagnostic  tests  to  pinpoint  problem 

Suggest  remedies 

Structure  discussions 

Examine  situation 

4.2.3  Discussion  of  Selected  Tools 

Tools  that  counter  cognitive  fragmentation  and  provide  data  to  the  monitors 

1 .  Database  Exploration.  In  this  architecture,  the  system  database  is  the 

central  repository  for  all  of  the  information  being  collected  and  all  of  the 
information  being  produced.  It  is  the  repository  for  all  of  the  team’s 
intermediate  and  final  data  products.  Because  the  database  is  aligned  with 
the  system’s  formal  ontology,  every  table  and  data  field  is  unambiguously 
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associated  with  domain  concepts.  Team  members  use  the  database 
exploration  tool  to  review,  retrieve,  and  deposit  database  information. 

Support  to  understandings: 

a.  Task  assessment.  Because  every  authorized  team  member  can  review  the 
information  in  the  database,  other  team  members  can  infer  what  tasks  are 
being  worked  on  providing  the  task  produces  intermediate  products 
deposited  in  the  database  and  the  team  members  know  what  information  is 
associated  with  the  product.  Team  members  can  infer  task  progress  by 
noting  the  rate  at  which  the  information  is  deposited.  They  can  infer 
problems  if  the  rate  of  deposit  is  too  slow.  In  addition,  they  can  infer  if 
progress  is  being  impeded  by  information  unavailability  if  they  know  that 
information  that  a  task  needs  to  retrieve  from  the  database  is  not  there. 

b.  Mutual  understanding.  By  reviewing  the  information  that  other  team 
members  produce,  team  members  can  infer  what  concepts  these  team 
members  are  addressing  and  can  judge  the  extent  to  which  they  agree  or 
disagree  with  these  concepts.  If  the  team  members  have  sufficient 
background  in  an  area,  they  can  judge  the  level  of  expertise  of  the  team 
members  producing  the  information. 

c.  Situation  Understanding.  As  the  repository  for  information  and  products, 
the  database  contains  information  about  the  current  situation  and  the 
evidence  supporting  that  information.  By  reviewing  the  database  contents, 
team  members  can  update  their  understanding  of  the  situation  and 
evidence,  and  can  judge  the  extent  to  which  the  information  is  incomplete 
or  imprecise. 

Data  Provided  to  Monitors.  The  database  transaction  records  reference  the 
information  that  team  members  deposit,  review,  and  extract  from  the  database.  These 
data  illuminate  how  well  team  members  are  keeping  up  with  schedule,  what  tasks  are 
being  worked  on  and  by  whom,  what  tasks  are  compete,  what  issues  are  being  deposited 
into  the  database,  and  the  state  of  intermediate  product. 

2.  DCODE  (Decision  Making  Constructs  in  a  Distributed  Environment). 
DCODE  seeks  to  “improve  the  ability  of  distributed  group  decision 
makers  to  evaluate,  share,  and  integrate  decision-relevant  information 
items,”  and  “to  reduce  the  time  and  effort  devoted  to  conflict  resolution 
and  consensus  building  in  reaching  an  overall  group  decision”  (DCODE 
brief).  DCODE  works  by  encapsulating  critical  meta-data  about  an 
information  item  in  easily  shared  Information  Objects  (IOBs).  Each  IOB 
labels  an  information  item  with  a  unique  identifier,  specifies  the  issue  it 
addresses,  and  notes  the  team  member  who  created  it.  The  IOB  also 
incorporates  critical  meta-data  on  the  information  item’s  credibility, 
timeliness,  effect  (extent  it  supports  or  undermines  a  position)  and 
importance  (weight  it  should  be  given).  IOBs  help  teams  consider  the  full 
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range  of  information  available  on  a  topic  and  helps  them  to  balance  the 
diversity  of  viewpoints  expressed  in  this  information,  taking  into  account 
the  credibility  of  the  information  source.  DCODE  maintains  an  audit  trail 
between  an  IOB  and  the  information  it  references,  enabling  all  team 
members  to  easily  review  the  original  source  information.  DCODE’s 
ability  to  capture  source  meta-data  is  important  part  of  the  Human- 
Focused  architecture.  SPAWAR  developed  DCODE  as  part  of  ONR’s 
Human  System’s  Department. 

Support  to  understandings: 

a.  Activity  awareness  (what  others  are  doing).  Because  each  IOB  references 
the  name  of  the  person  who  created  it,  team  members  can  determine  what 
information  others  on  the  team  are  reviewing,  and  so  infer  the  issues  that 
they  are  working  on. 

b.  Mutual  understanding  (other’s  current  understandings).  Each  IOB 
summarizes  its  creator’s  evaluation  of  the  information  item,  including  that 
creator’s  views  on  the  source’s  credibility,  on  the  item’s  impact  on  an 
issue  to  be  evaluated,  and  on  the  importance  of  the  issue.  The  set  of  IOB  s 
created  by  a  team  member  reflects  that  team  member’s  understandings  on 
team  issues.  By  reviewing  the  information  item  that  the  IOB  summarizes, 
other  team  members  can  determine  the  extent  to  which  their  views  about 
the  information  align  with  the  views  of  the  IOB’s  creator. 

c.  Situation  understanding  (evidence).  The  IOBs  summarize  the  evidence  for 
and  against  a  team  position  as  needed  to  understand  the  diversity  of 
viewpoints  on  an  issue.  IOBs  as  a  set  incorporate  three  drivers  in 
evaluating  a  position  from  the  information  about  that  position:  the  number 
of  information  items  with  a  particular  position,  the  strength  of  the 
information  source’s  arguments  (importance  in  the  IOB),  and  the 
credibility  of  each  source. 

Data  Provided  to  Monitors.  DCODE  records  team  members’  capsule  summaries 
of  documents.  These  records  therefore  can  convey  the  issues  a  team  member  has  been 
addressing,  and  can  provide  these  summaries  to  the  monitors. 

3.  EWALL  helps  distributed  team  members  “collect,  organize  and  view 

graphical  and  contextual  information,  to  comfortably  collaborate  in  large, 
distributed  and  decentralized  teams,  as  well  as  to  maintain  their  individual 
ways  of  working  (MIT  brochure).  EWALL  can  extract  information  from 
diverse  sources,  structure  the  information  in  a  standard  card  format,  and 
send  the  information  to  team  members  that  need  it.  Team  members  can 
integrate  each  card  in  their  input  stream  into  their  personal  viewing  space. 
When  they  do  so,  EWALL  infers  the  team  member’s  interests  in  the 
information  and  also  infers  how  the  newly  integrated  information  relates 
to  other  information  that  that  team  member  is  working  with.  Based  on  its 
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inferences,  EWALL  can  automatically  route  information  to  team  members 
who  need  it,  whether  the  information  is  retrieved  from  sources  outside  of 
the  team  or  produced  within  the  team.  Because  each  team  member  can 
view  other  member’s  card  layouts,  EWALL  strongly  supports  team 
transparency,  helping  each  team  member  know  what  other  team  members 
are  working  on.  EWALL  is  sponsored  by  ONR’s  Human  System’s 
Department. 

Support  to  understandings.  Because  of  its  ability  to  increase  team  transparency, 
EWALL  can  help  unify  team  members’  understandings  in  many  areas.  Areas  where  it 
provides  the  most  direct  support  are: 

a.  Activity  awareness  (what  others  are  doing).  EWALL  allows  each  team 
member  to  view  other  member’s  EWALL  work  space.  Thus,  it  enables 
each  team  member  to  directly  see  what  other  team  members  are  working 
on.  Because  EWALL  can  deposit  team  member’s  information  products 
into  a  common  team- viewable  data  repository,  it  helps  team  members 
infer  what  people  are  working  on  by  reviewing  what  others  have 
produced. 

b.  Task  assessment  (tasks  being  worked  on  and  progress).  EWALL  supports 
this  indirectly,  as  a  by-product  of  team  transparency.  If  team  members 
know  the  work  required  to  support  team  tasks,  then  team  members  can 
infer  what  tasks  are  being  worked  on  by  knowing  what  each  team  member 
is  doing.  If  team  members  know  the  steps  required  to  complete  a  task,  then 
reviewing  the  work  enables  people  to  infer  task  progress.  Note  that 
EWALL  by  itself  is  not  sufficient  for  task  assessment.  This  assessment 
requires  knowing  not  only  what  people  are  working  on,  but  also  knowing 
what  work  is  required  for  each  task. 

c.  Mutual  understanding  (others  current  understandings).  Like  task 
assessment,  support  to  mutual  understanding  is  another  by-product  of  team 
transparency.  By  reviewing  how  a  team  member  is  interpreting  received 
information,  other  team  members  can  infer  the  extent  to  which  this 
interpretation  aligns  with  their  own.  By  reviewing  what  another  team 
member  is  doing,  other  team  member’s  can  sometimes  infer  the  extent  to 
which  that  team  member’s  views  on  team  goals  and  plans  align  with  their 
own.  In  addition,  by  routing  team  member’s  work  products  to  other  team 
members,  EWALL  helps  team  members  review  these  products  and  thus 
helps  them  determine  the  extent  to  which  the  views  on  the  forwarded  cards 
align  with  their  own  views 

Data  Provided  to  Monitors.  EWALL  encapsulates  information  onto  a  set  of 
cards,  records  the  source  of  the  information,  and  routes  the  information  to  team  members 
that  might  use  it.  It  also  records  who  is  interested  in  the  information  and  how  they  are 
relating  new  information  to  older  material.  Therefore  EWALL  can  record  the  concepts 
that  a  team  member  receives,  the  source  of  these  concepts,  the  concepts  the  team  member 
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is  working  on,  the  concepts  that  the  team  member  is  producing,  and  the  destination  of 
concepts  produced.  EWALL  is  a  powerful  source  of  information  to  the  concept  flow 
monitor. 

4.  COBRA  EVM  provides  information  on  actual  task  and  plan  status.  To  use 
the  tool,  the  team  enters  plan  estimates  for  schedule  and  labor  for  each 
milestone,  each  task  and  for  all  performance  measures.  In  addition,  the 
planner  specifies  the  “earned  value”  accrued  when  each  milestone  is 
achieved.  At  various  time  intervals,  team  members  enter  actual  data  for 
labor  and  milestone  achievement.  The  system  then  calculates  the  earned 
value  of  the  work  performed  given  the  milestones  achieved  and  reports 
cost  and  schedule  variances.  Cost  Variance  (CV)  is  the  difference  between 
the  planned  and  actual  resource  usage  for  an  element  of  work.  A  negative 
variance  means  that  more  money  was  spent  for  the  work  accomplished 
than  was  planned.  Schedule  Variance  (SV)  is  the  difference  between  the 
budgeted  cost  of  work  performed  and  the  earned  value  for  an  element  of 
work.  Any  difference  is  called  the  Schedule  Variance.  The  tool  will 
provide  graphs  of  the  overall  program  or  individual  tasks  that  may  be 
underperforming.  Cobra  will  also  calculate  the  Cost  Performance  Index 
(CPI)  and  the  Schedule  Performance  Index  (SPI).  Using  these  indexes, 
team  members  can  calculate  final  cost  (budgeted  cost  /CPI)  and  final 
project  duration  (planned  project  duration/time  based  SPI).  The  purpose  of 
earned  value  management  is  to  integrate  budget  and  performance 
measures  to  determine  actual  project  status. 

Support  to  understandings.  EVM  collects  data  on  task  performance  to  support 
project  tracking  for  reporting  to  management  and  clients.  It  provides  excellent  support  to 
activity  awareness  and  task  assessment,  but  only  with  the  granularity  needed  for  project 
support. 

a.  Activity  awareness.  By  tracking  task  progress,  EVM  can  help  team 
members  be  aware  of  what  team  members  are  doing,  how  well  they  are 
keeping  up  with  schedule,  and  the  extent  to  which  the  task  is  proceeding  in 
accordance  with  the  plan. 

b.  Task  assessment.  This  is  the  purpose  of  EVM.  It  provides  visibility  into 
the  tasks  being  worked  on,  progress,  possible  problems,  and  resource 
needs.  Note  because  it  does  not  do  this  continuously  on  an  hour  by  hour 
basis,  it  should  be  supplemented  by  other  methods  that  do  provide  more 
continuous  monitoring. 

Data  Provided  to  Monitors.  Team  members  manually  enter  task  progress  and 
the  plan  into  EVM,  which  then  computes  the  variance  of  progress  with  the  plan.  EVM 
can  provide  these  performance  data  to  the  cognitive  monitors  to  support  the  coping  and 
task  assessment. 

Tools  that  counter  cognitive  fragmentation  and  provide  data  to  the  monitors 
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5.  The  Collaboration  Advizor  helps  teams  diagnose  and  fix  cognitive 
problems.  To  use  the  tool,  each  team  member  answers  a  short 
questionnaire  about  the  collaboration  environment,  team  behaviors,  task 
accomplishments  and  team  member  understandings.  Each  of  these 
questions  is  diagnostic  of  team  problems  in  one  or  more  of  twelve 
“cognitive  enablers” — areas  of  understanding  that  are  important  for  team 
members  to  work  together  effectively.  The  tool  then  assembles  the 
individual  responses  to  create  a  “team  view.”  This  view  summarizes  team 
strengths  and  weaknesses  for  each  of  the  twelve  knowledge  areas,  and  can 
recommend  actions  that  address  team  weaknesses.  By  showing  how  team 
members  (anonymously)  responded  to  each  of  the  questions  the 
Collaboration  Advizor  team  view  can  reveal  where  team  members  agree 
and  disagree  about  their  understandings  of  team  issues.  In  addition,  the 
Advizor  can  track  team  changes  over  time,  so  that  members  can  see 
whether  actions  that  they  have  taken  to  address  problems  are  working,  or 
whether  the  team  needs  to  try  alternative  actions.  Evidence  Based 
Research  developed  the  Collaboration  Advizor  as  work  sponsored  by 
ONR’s  Human  System’s  Department. 

Support  to  understandings: 

a.  Activity  awareness  (accordance  with  plan  and  goals).  The  Collaboration 
Advizor  asks  many  questions  intended  to  reveal  the  extent  that  team 
members  understand  team  goals  and  plans.  It  can  help  make  team 
members  aware  of  the  risk  that  some  team  members  may  fail  to  act  in 
accordance  with  team  plans  and  goals. 

b.  Task  assessment  (possible  problems  and  impact).  The  Advizor  probes  for 
problems  that  team  members  may  have  in  carrying  out  their  tasks  and  in 
supporting  each  other.  For  example,  it  asks  about  whether  team  members 
have  the  experience  and  background  to  carry  out  their  tasks,  and  whether 
they  understand  task  dependencies  as  needed  to  adjust  their  tasks  should 
other  tasks  be  having  problems. 

c.  Mutual  understanding  (extent  others  agree  or  disagree).  Support  to  mutual 
understanding  is  one  of  the  Advizor’s  principal  objectives.  The  Advizor’s 
team  view  displays  the  number  of  people  agreeing  or  disagreeing  with 
each  of  the  questions  asked,  enabling  all  team  members  to  see  the  degree 
of  agreement  and  disagreement  on  numerous  issues.  For  example,  the 
team  view  might  show  that  five  of  eight  team  members  agree  with  the 
statement  “There  are  no  specified  criteria  for  mission  success.”  In  such 
cases,  those  members  who  believe  that  there  are  such  criteria  should 
explain  them  to  the  others. 

Support  to  Monitor  Follow-up.  The  Collaboration  Advizor  is  a  premier  tool  for 
following  up  monitor  cues.  It  can  help  team  pinpoint  the  underlying  cognitive  problems 
responsible  for  the  monitors’  output  readings,  and  it  can  suggest  remedies.  Furthermore, 
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by  creating  a  set  of  forms  that  encapsulate  team  issues  and  illuminate  team  member 
differences,  the  Advizor  can  help  structure  the  follow-up  discussions. 

6.  Mind  Manager.  This  tool  helps  teams  organize  their  ideas  in  a 
brainstorming  environment.  Mind  Manager  organizes  ideas  hierarchically, 
showing  the  relationship  of  elaborations  or  detail  thinking  on  an  issue.  It 
displays  this  organization  in  a  flexible  way  that  encourages  discussion. 

Support  to  understandings: 

a.  Mutual  understanding.  Mind  Manager’s  ability  to  help  structure  and  depict 
ideas  helps  all  team  members  be  aware  of  what  other’s  views  are  and 
helps  them  discuss  areas  of  agreement  and  disagreement. 

Support  to  Monitor  Follow-up.  Mind  Manager’s  ability  to  help  structure 
discussions  helps  teams  explore  team  issues  and  identify  remedies.  For  example,  team 
members  can  describe  specific  behaviors  that  substantiate  or  refute  the  significance  of  the 
issues  raised  by  the  monitors.  If  the  behaviors  are  substantiated,  the  team  might  discuss 
the  root  cause  of  the  behaviors  and  then  brainstorm  on  methods  for  addressing  the 
problem. 

7.  IBIS  (Issue-Based  Information  System).  IBIS  is  a  systematic  method  for 
developing  consensus  on  solutions  to  “wicked”  problems,  drawing  on  the 
work  of  Kuntz  and  Rittel  in  the  1970s  (Kuntz  &  Rittel,  1972).  Wicked 
problems  are  problems  in  which  the  nature  of  the  problem  is  not  clear  at 
first,  but  can  emerge  as  the  group  discusses  the  pro’s  and  con’s  of  various 
proposed  problem  solutions.  There  are  several  different  tools  that  support 
the  IBIS  process  (e.g.,  gIBIS,  Compendium),  one  of  which  would  be 
selected  to  implement  the  architecture.  Each  of  these  tools  helps  the  group 
structure  its  dialog  into  a  set  of  Questions,  Ideas,  and  Arguments.  The 
question  states  the  problem  as  a  question  to  be  answered.  The  ideas  are 
possible  answers  to  the  question,  and  the  arguments  are  the  reasons  for 
supporting  or  objecting  to  an  idea 

(http://www.touchstone.com/tr/wp/IBIS.html).  Generally,  the  IBIS  process 
helps  teams  explore  the  nature  of  the  problem,  encouraging  a  full  diversity 
of  viewpoints,  as  in  brainstorming.  The  process  then  also  helps  the  teams 
consolidate  these  viewpoints  as  required  to  support  development  of 
consensus. 

Support  to  understandings.  An  IBIS  tool  helps  team  members  be  aware  of  each 
other’s  viewpoints  and  the  reasons  for  these  viewpoints. 

a.  Mutual  understanding  (others  current  understandings).  In  an  IBIS-structured 
dialog,  team  members  share  their  current  interpretations  of  possibly  poorly 
defined  team  goals,  state  their  ideas  on  how  to  achieve  these  goals,  and 
articulate  their  views  on  the  strengths  and  weaknesses  of  proposed 
solutions.  By  making  these  views  explicit,  IBIS  helps  team  members  know 
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what  each  other’s  views  are,  and  equally  important,  helps  team  members 
know  each  other’s  reasons  for  their  views. 

b.  Mutual  understanding  (Extent  others  agree  of  disagree).  By  making  explicit 
different  viewpoints  on  an  issue,  IBIS  can  also  clarify  where  people  agree 
or  disagree,  and  if  they  disagree,  can  help  clarify  why. 

Support  to  Monitor  Follow-up.  IBIS  helps  structure  team  discussions.  When 
team  members  are  debating  the  nature  of  the  team’s  problems,  IBIS  can  help  collect  and 
compare  team  member  views  on  what  the  source  of  the  problem  is,  the  different  ways 
that  the  team  can  address  the  problem,  and  the  strengths  and  weaknesses  of  each  of  these 
different  ways. 


8.  Visual  Links.  Visual  Links  is  an  application  that  helps  people  understand 
the  content  of  the  database  and  explore  the  relationships  among  the  data. 

Support  to  understandings: 

a.  Situation  understanding.  When  the  database  stores  information  about  the 
situation,  Visual  Links  can  support  all  facets  of  situation  understanding.  It 
supports  surface  understanding  by  help  people  review  the  facts  about  the 
situation — the  nature  of  the  situation  entities  and  situation  events.  It 
support  understanding  how  these  entities  impact  each  other  by  revealing 
relationships  in  the  situation,  thereby  helping  people  make  forecasts  and 
predictions.  It  helps  people  understand  the  evidence  by  linking 
conclusions  to  supporting  and  contradicting  observations.  It  helps  people 
understand  uncertainties  by  clarifying  gaps  and  imprecision  in  the 
situation  information. 

Support  to  Monitor  Follow-up.  When  team  problems  stem  from  conflicting 
incomplete  concepts  about  the  situation,  Visual  Links  can  help  team  members  examine 
what  is  known  about  the  situation.  When  team  problems  manifest  themselves  as 
incomplete  or  poor  quality  intermediate  products,  Visual  Links  can  help  the  team  analyze 
their  product  to  determine  areas  that  require  improvement. 

General  Communication  Tools 

Communication  tools  include  the  telephone,  e-mail,  chat,  video-teleconferencing, 
broadcasts,  web-posting  and  other  ways  for  people  to  convey  information  to  each  other. 
We  include  them  in  this  discussion  because  they  contribute  to  numerous  aspects  of 
teamwork. 

Communication  tools  are  pervasive  now,  and  will  be  part  of  any  distributed 
collaboration  architecture.  We  include  them  for  completeness,  because  they  are  important 
for  preventing  cognitive  fragmentation,  for  feeding  the  cognitive  monitors,  and  for 
enabling  teams  to  address  team  problems. 
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From  the  perspective  of  cognitive  fragmentation,  communication  tools  may  be 
classified  by  their  ability  to  provide  feedback  for  understanding  what’s  being  said.  In 
normal  conversation,  speech  occurs  in  segments.  After  a  speaker  finishes  a  thought,  the 
information  recipient  signals  the  extent  to  which  the  thought  is  understood.  The  speaker 
upon  receiving  the  signal  either  continues  if  the  signal  indicated  understanding,  or 
rephrases  or  elaborates  if  the  signal  indicated  an  undesired  level  of  understanding. 
Occasionally  the  receiver  will  ask  for  clarification.  Usually  however  the  understanding  of 
a  thought  it  expressed  through  body  language,  facial  expressions,  tone  of  voice,  or  short 
informal  words  or  phrases. 

Because  of  the  importance  of  these  understanding  cues,  the  ability  of  the  medium 
to  transmit  these  cues  impacts  the  ease  with  which  meaning  can  be  transferred.  The 
medium  most  effective  for  transfer  of  meaning  is  face-to-face  conversation,  since  it  can 
transfer  all  of  these  cues.  Second  are  mediums  that  can  convey  body  language,  such  as 
VTC.  Third  are  voice  communications,  because  they  convey  tone  of  voice.  Next  are  text 
transmissions  with  immediate  feedback.  After  that  is  text  transmission  without  immediate 
feedback  but  in  which  feedback  is  very  easy,  such  as  e-mail.  Last  are  medium  in  which 
feedback  is  not  facilitated  (and  sometimes  not  possible)  such  as  broadcasts  or  web  pages. 

In  addition  to  facilitating  understanding  feedback,  communication  tools  vary  in 
their  ability  to  convey  relationships  within  a  concept.  For  this  use,  medium  that  can 
present  pictures  are  helpful.  Face  to  face  with  white  board  is  most  effective,  for  it  permits 
diagrams  as  well  as  providing  immediate  understanding  feedback.  For  distributed  teams, 
shared  applications  that  enable  diagrams  with  immediate  feedback  can  be  very  helpful. 

In  our  human-centric  architecture,  communication  tools  provide  data  to  the 
cognitive  monitors.  Text  can  be  directly  input  into  the  content  analysis  tools,  and  voice 
can  be  input  after  being  transcribed  into  text.  The  content  analysis  tools,  such  as  Latent 
Semantic  Analysis  or  Information  Extraction,  can  infer  from  this  input  the  concepts  being 
communicated  and  their  source,  and  can  often  infer  the  destination  of  the  concepts. 

4.3  Reference  Documents 

This  section  addresses  three  reference  documents  that  the  cognitive  monitors 
need.  It  describes  each  of  these  documents  and  summarizes  its  contribution  to  the 
cognitive  monitors. 

The  three  documents  are  the  team’s  plan,  a  plan  supplement  that  enumerates 
issues  that  the  plan  should  address,  and  the  team’s  expertise  map.  One  of  these 
documents,  the  team’s  plan,  should  be  a  part  of  nearly  every  team’s  arsenal.  The  others, 
the  issues  to  be  addressed  and  the  expertise  map,  will  also  be  valuable  to  most  teams. 

The  plan  specifies  the  team  goals  and  describes  a  set  of  tasks  for  achieving  these 
goals.  It  describes  the  goals  in  a  manner  intended  to  help  team  members  understand 
commander’s  intent.  Often,  it  will  describe  the  desired  end  state. 

In  addition  to  specifying  the  goals  and  tasks,  plans  specify  resources,  schedules 
with  milestones,  dependencies,  contingencies,  and  performance  measures.  Generally,  a 
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plan  specifies  who  is  responsible  for  carrying  out  the  task,  and  the  information  required 
and  the  equipment  and  materials  allocated.  The  plan  specifies  when  the  tasks  are  to  be 
done  and  what  should  be  accomplished  at  specified  times  (the  milestones).  Plans  state  the 
other  tasks  and  goals  that  depend  on  the  task.  Because  plans  often  cannot  ensure  success 
under  every  possible  circumstance,  the  plan  specifies  those  circumstances  under  which  it 
may  not  work  and  defines  the  alternative  actions  to  be  carried  out  under  various  other 
circumstances  (the  contingencies).  Finally  a  plan  may  specify  performance  measures  for 
measuring  progress  as  the  plan  proceeds. 

In  the  human-centric  architecture,  the  plan  is  formally  documented  in  the  Earned 
Value  Management  application.  This  system  can  export  it  to  the  monitors. 

The  plan  documentation  is  required  to  support  the  coping  monitor  and  the 
evaluative  parts  of  the  task  assessment  monitor.  Because  the  plan  communicates  to  the 
monitors  when  tasks  should  be  accomplished,  if  the  monitor  knows  when  tasks  are  being 
completed,  it  can  then  determine  whether  people  are  falling  behind  (coping  meter  input), 
can  determine  plan  progress  with  respect  to  schedule,  and  can  determine  scheduled  tasks 
that  are  not  being  addressed. 

Issue  enumeration  supplement.  In  our  description  of  the  concept  flow  monitor, 
we  distinguished  between  the  plan  and  the  list  of  issues  to  be  addressed  in  carrying  out 
the  plan.  The  plan  itself  enumerates  a  series  of  steps  to  take  to  achieve  a  goal.  In  contrast, 
the  issue  enumeration  lists  issues  that  should  be  addressed  when  carrying  out  the  plan.  In 
the  case  of  disaster  relief,  the  issues  might  include  determine  the  populations’  needs  at 
various  locations,  determining  the  urgency  of  these  needs,  determining  needed  medical 
supplies,  and  determining  how  to  transport  the  supplies  to  the  people  who  need  them.  The 
plan  issue  supplement  enumerates  the  issues  that  need  to  be  addressed. 

This  supplement  is  important  to  the  concept  flow  monitor.  It  defines  the  concepts 
that  the  plan  specifies  should  be  addressed,  provides  the  materials  that  LSA  needs  to 
learn  to  recognize  these  concepts,  and  provides  the  reference  that  enables  the  concept 
flow  monitor  to  judge  whether  the  team  is  addressing  the  concepts  it  needs  to. 

Team  expertise  map.  This  map  specifies  the  issues  that  team  members  are  expert 
at  or  are  responsible  for  handling.  It  enables  the  concept  flow  monitor  to  note  the  extent 
that  team  members  are  drawing  on  team  expertise  when  they  address  an  issue. 
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5.  ARCHITECTURE 

The  architecture  provides  the  connectivity  and  information  management  needed 
to  support  net-centric  operations.  This  peer-to-peer  agent-based  architecture  facilitates 
information  flow,  helps  integrate  a  collaboration  tool  suite  tailored  to  team  needs,  and 
gathers  and  manages  the  cognitive  node  monitors.  Its  use  of  the  Groove  peer-to-peer 
application  provides  for  robust  connectivity,  supporting  even  team  members  at  nodes 
behind  firewalls,  with  low  bandwidth  access,  or  having  interrupted  access.  Its  use  of 
COABS  for  agent  management  facilitates  use  of  intelligent  that  help  manage  information 
flow  and  integrate  applications. 

Development  of  this  architecture  is  extremely  low  risk.  Most  of  the  architecture  is 
currently  operational  at  the  NPS  Gigalab  and  EBR  war  room.  Additional  upgrades  will 
link  the  NPS  and  EBR  systems  and  integrate  agents  for  managing  the  data  flows  that  the 
cognitive  node  monitors  need. 

The  architecture  is  layered  and  geographically  distributed.  Every  peer-to-peer 
collaborative  node  operates  on  the  top  of  tactical  wireless  mesh  network,  such  as  one  the 
described  on  Figure  13,  which  is  integrated  globally  with  the  EBR  War  Room 
environment  via  the  satellite  or  high-speed  wired  communications  within  the  Global 
Information  Grid  (GIG)  services  in  a  way  similar  to  Figure  14. 


Figure  13.  NPS  Tactical  Wireless  Testbed  for  Sensor-Decision  Maker-Tactical 
Operation  Centers  Collaboration  Studies 
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Information  Sharing  and  Collaborative  Action/  Cognitive  node  support: 
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Figure  14.  Network  Level  Integration  of  EBR  War  Room  with  NPS  tactical 

Collaboration  Testbed 

Such  infrastructure  enables  cognitive  node  connectivity  at  layers  1-7  in 
accordance  with  OSI  communication  stack  model.  The  Groove  clients  reside  at  layer  7  of 
OSI  communication  stack  at  every  sensor-decision  maker  grid  node.  They 
correspondingly  enable  peer-to-peer  collaborative  application  flow  across  the  grid 
encapsulating  shared  and  situational  awareness  views  (Figure  15) 
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Figure  15.  Situational  Awareness  View  Shared  by  Multiple  DMs  and  Integrated  with 
Groove  Peer-to-Peer  Collaborative  Environment. 

Middleware  agent  services  to  collaborative  clients  ensure  seamless  integration  of 
different  applications  residing  on  the  War  Room  server  and  across  the  tactical  grid  nodes. 
The  multi-agent  middleware  belongs  to  the  lower  level  of  application  layer  hierarchy 
within  the  proposed  architecture: 

OSI  Application  Layer: 

-Cognitive  tier  (the  cognitive  node  monitors) 

-Peer-to-Peer  collaborative  applications  tier  (Groove,  EWall) 

-Situational  Awareness  agents  (NPS  CoABS  model) 

OSI  Networking  Layers  (1-6). 

The  agents  allow  any  application  on  behalf  of  decision  maker  to  register  on  the 
grid  and  also  to  establish  sensor-decision  maker  interfaces  with  Groove  and  EWall  in 
order  to  ensure  collaborative  process  monitoring  up  to  the  cognitive  level.  Figure  16 
illustrates  an  example  of  agent  architecture  for  integrating  reconnaissance  and 
surveillance  applications  in  the  collaborative  environment.  The  integration  mechanism  is 
based  on  the  CoABS  agent  services  model. 
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Figure  16.  Multi-Agent  Architecture  for  Application- Decision  Maker  Integration 

in  Shared  Collaborative  Environment 

By  providing  for  ubiquitous  application-decision  maker  integration  the  proposed 
multi-agent  architecture  allows  unifying  high  level  cognitive  events  monitoring  (activity 
awareness,  task  assessment,  mutual  understanding,  situational  understanding)  with  lower 
level  networking  events  awareness. 

Figure  17  illustrates  the  model  of  lower  (physical  networking)  level  awareness 
brought  up  to  shared  collaborative  environment  via  agents,  which  will  be  expanded  in  the 
proposed  human-centric  architecture  to  cognitive  event  “gauges”.  Note  that  top  level 
cognitive  gauges  in  the  upper  left  hand  corner  of  Figure  17.  These  stop  lights  are  the  ones 
shown  and  explained  in  Figure  4  in  Chapter  3. 
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Figure  17.  Lower  Level,  Physical  Networking  Events  Monitoring 
via  the  Cognitive  Node  Agents. 

In  Figure  17  the  levels  of  collaborative  node  monitoring  include  network 
(throughput,  responsiveness,  and  packet  loss  gauges)  and  ISR  applications  (video  option 
above).  In  the  planned  human-centric  architecture  the  menu  of  info,  video,  and  network 
monitoring  options  in  the  cognitive  node  agent  frame  will  be  expanded  by  the  cognitive 
enablers  described  in  the  previous  sections. 

In  figure  17  the  alert  window  on  the  left  is  the  agent  response  to  the  application 
event,  namely  video  motion  detection.  In  our  proposed  architecture  for  cognitive  node 
monitoring  similar  events  will  be  generated  by  the  agent,  but  at  the  higher  level  of 
cognitive  enablers,  i.e.  problems  in  task  (task  assessment),  uncertainty  (situation 
understanding),  etc.  This  is  indicated  by  the  stop  lights  at  the  top  of  Figure  17. 

Once  the  cognitive  enabler  alert  is  issued,  the  shared  situational  awareness  view  is 
captured  and  transferred  to  the  EWALL  for  subsequent  DECODE  -based  evaluation  of 
importance,  relevance,  and  other  conclusions.  It  will  be  done  in  a  way  similar  to  the  ISR 
application  events  association  via  the  EWall  as  presented  in  Figure  18. 
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Figure  18.  EWALL  Integration  with  Application  Event  Awareness. 


In  figure  18  each  row  corresponds  with  agent-decision  maker  events,  which  are 
captured  over  time  for  an  immediate  or  subsequent  evaluation.  The  situational  awareness 
view  is  the  disclosure  of  one  if  the  event  cards.  Once  the  cognitive  enablers  are  added  to 
agent  monitoring  levels,  they  will  appear  in  the  evaluation  view  accordingly. 
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6.  BENEFITS  TO  NAVY 

The  new  military  doctrines  of  Network  Centric  Warfare  and  the  Navy’s  ForceNet 
are  transforming  our  military  into  a  much  more  agile  force,  able  to  comprehend  the 
situation  and  the  commander’s  intent  and  quickly  seize  opportunities  and  thwart  risks 
through  a  broad  diversity  of  often  multi-national  operations.  These  doctrines,  though  still 
evolving,  are  now  starting  to  be  put  into  practice.  Both  Operation  Enduring  Freedom  in 
Afghanistan  and  Operation  Iraqi  Freedom  in  Iraq  incorporated  many  elements  of 
Network-Centric  Warfare. 

The  intellectual  foundation  of  these  agile  and  responsive  forces  is  the  ability  to 
quickly  create  a  cognitively  cohesive  team,  where  every  team  member  understands  how 
his  work  fits  with  the  work  that  others  are  doing,  understands  others’  responsibilities  and 
areas  of  expertise,  knows  what  others  are  doing  and  how  it  contributes  to  the  team  goals, 
is  aware  of  what  others’  views,  and  knows  where  team  members  agree  and  disagree  with 
each  other. 

It  is  not  easy  to  quickly  form  and  connect  ad-hoc  teams  with  the  special  mix  of 
disciplines  and  backgrounds  needed  to  handle  rapidly  evolving  situations.  It  is  even  more 
difficult  to  ensure  that  the  team’s  members  can  understand  and  leverage  each  others’ 
perspectives  to  create  high  quality  situation  assessments  and  plans.  The  human- focused 
architecture  and  cognitive  monitors  that  this  STTR  will  provide  will  increase  team 
transparency,  helping  all  team  members  achieve  the  common  understandings  necessary 
for  effective  teamwork.  The  architecture  and  system  will  also  measure  team 
performance,  as  needed  to  cue  team  members  to  possible  problems  that  the  team  should 
address. 

The  cognitive  monitors  can  also  instrument  team  performance  measurement, 
enabling  team  performance  data  to  be  collected  automatically  and  non- intrusively.  This 
capability  will  enable  the  Navy  to  systematically  improve  team  performance  by  knowing 
the  tools,  training,  and  organizations  that  best  enhance  teamwork. 
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